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I^ABSTRACT 

Many  communications  and  Radar  receivers  must  process  data  over  a  very  wide  band,  which  requires  either  high-rate  analog-to-digital  conveners  (ADCs) 
or  mukichanncl  receivers.  The  information  content  of  that  wideband  data,  however,  is  often  sparse  in  some  basis.  Analog- to^Informaiion  fA21)  receivers 
exploit  this  sparseness  in  both  the  digital  and  analog  domains  by  non-adaptively  spreading  the  signal  energy  (analog)  and  using  digital  signal  processing  to 
recover  the  signal  from  an  ADC  sampling  at  a  sub-Nyquist  rate,  A  subsampled  ADC  implies  the  use  of  fewer  recciv^er  channels  or  less  expensive,  lower- 
rate  devices.  This  report  documents  the  signal  processing  techniques  for  such  receivers  developed  by  the  MIT  Lincoln  Laboratory/GMR  Research  and 
Technology  team  in  the  study  phase  of  the  A21  program.  We  have  developed  two  new  A21  signal  processing  methods,  both  significantly  outperforming 
compressed  sensing  (CS)  techniques  currently  in  the  literature,  which  typically  fail  when  signals  occupy  more  than  15  20%  of  the  downsampled  band.  One 
of  our  methods.  Nonlinear  Affine  processing  (NoLaff),  uses  a  nonlinear  front-end  to  spread  signal  energy  before  the  sub-Nyquist  ADC,  and  uses 
hypothesis  testing  to  reconstruct  the  signal.  In  simulations,  this  technique  has  shown  that  it  can  reconstruct  wideband  signals  occupying  up  to  72%  of  the 
dowmsampicd  basis.  It  is  also  much  less  sensitive  to  the  difficulties  CS  has  detecting  signals  with  large  magnitude  variation  in  the  compressible  basis.  Our 
other  method,  called  Variable  Projection  and  Unfolding  (VPU),  spreads  the  signal  energy  using  random  linear  projections  similar  to  those  used  in 
compressed  sensing,  but  is  able  to  reconstruct  signals  occupying  nearly  100%  of  the  downsampled  basis.  VPU  achie%'es  this  using  a  technique  similar  to 
matching  pursuit;  the  key  difference  being  that  VPU  searches  over  blocks  of  consecutive  columns  rather  than  one  column  at  a  time.  Performance  bounds 
for  NoLaff,  VPU  and  traditional  compressed  sensing  algorithms  are  also  presented,  supporting  our  experimcrttal  results.  We  also  present  a  difierenl,  daia- 
adaptive  method  for  subsamplcd  signal  recovery:  the  use  of  dynamical  systems.  Dynamical  systems  provide  a  natural  model  for  A2I  receivers  due  to  their 
randomization  and  memory  properties.  When  driven  by  an  unknown  input  signal  that  is  sparse  in  some  known  basis,  a  dynamical  system  can  use  an 
ordinary  differential  equation  to  reconstruct  the  input  signal.  We  present  a  description  of  dynamical  system  implementation,  as  well  as  some  initial 
qualitative  results.  All  three  techniques  show  promise  for  use  in  future  imelligencc,  surveillance  artd  reconnaissance  systems. 
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ABSTRACT 


Many  coniiiuink'ations  and  Radar  receivers  must  process  data  over  a  very  wide  band,  whir:h  requires 
either  high-rate  aiialog-to-digital  converters  (ADCs)  or  imdtichaiinel  receivers.  The  information 
content  of  that  vrideband  data,  however,  is  often  sparse  in  some  basis.  Analog- to-Inforination  (A2I) 
receivers  exploit  this  sparseness  in  both  the  digital  and  analog  domains  by  non-adaptively  spreading 
tlie  signal  energy  (analog)  and  tising  digital  signal  processing  to  recover  the  signal  from  an  .\DC 
sampling  at  a  sub-Nyquist  rate.  A  subsampled  ADC  implies  the  use  of  fewer  receiver  channels 
or  less  expensive,  lower-rate  devices.  This  report  documents  the  signal  processing  techniques  for 
such  receivers  developed  by  the  MIT  Lincoln  Laboratory /G MR  Research  and  Technology'  team  in 
the  study  phase  of  the  A2I  program.  \\^  have  developed  two  new  .-121  signal  processing  methods, 
both  significantly  outperforming  compressed  sensing  (CS)  techniques  currently  in  the  literature, 
which  typically  fail  when  signals  occupy  more  than  15-20%  of  the  downsampled  band.  One  of 
our  methods,  Nonlinear  .4ffine  processing  (NoLAff),  uses  a  nonlinear  front-end  to  spreatl  signal 
energy  before  the  sul>-Nyciuist  ADC,  and  uses  hypothesis  testing  to  reconstruct  the  signal,  hi 
simulations,  this  technique  has  shown  that  it  can  reconstruct  widelaand  signals  occupying  up  to 
72%  of  the  downsampled  basis.  It  is  also  much  less  sensitive  to  the  difficulties  CS  has  tletecting 
signals  with  large  magnitude  variation  in  the  compressible  basis.  Our  other  method,  called  Variable 
Projwtion  and  Unfolding  (\TU),  spreads  the  signal  energy  using  random  linear  projections  similar 
to  those  usmI  in  compressed  sensing,  but  is  able  to  reconstruct  signals  occupying  nearly  100%)  of 
the  downsampled  basis.  \TU  achieves  this  using  a  technique  similar  to  matching  pursuit;  the  key 
difference  lieing  that  V’PU  searches  over  blocks  of  consecutive  columns  rather  than  one  column  at  a 
time.  Performance  bounds  for  NoLAff,  VPU  and  tr<iditional  compressed  sensing  algorithms  are  also 
presented,  supporting  our  experimental  results.  W'e  also  present  a  different,  data-adaptive  method 
for  siibsampled  signal  recovery:  the  use  of  dynamical  systems.  Dynamical  systems  provide  a  natural 
model  for  .421  receivers  due  to  their  randomization  and  memory  properties.  When  driven  by  an 
imknow'ii  input  signal  that  is  sparse  in  some  kuow'ii  basis,  a  dyrianiical  system  can  use  an  ordinary 
differential  ecpiation  to  reconstrvict  the  input  signal.  W'e  present  a  description  of  dynamical  system 
iin{)lementatiou,  as  well  as  some  initial  qualitative  results.  .411  three  techniques  show  promise  for 
use  ill  future  iiitelligence,  surveillance  and  reconnaissance  systems. 
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1.  INTRODUCTION 


Comniuiiications  and  Radar  receivers  are  frequently  tasked  with  processing  very  wideband 
signals.  In  many  rases,  the  information  content  of  the  very  wideband  signal  is  compressible  in  some 
basis,  equatitjg  to  the  signal  having  a  sparse  representation  in  the  basis  in  which  it  is  compressible. 
Analog-to-Information  (A2I)  receivers  exploit  a  signal's  sparse  representation  by  non-adaptively 
spreading  the  RF  IF  signal  and  then  recovering  the  original  signal  from  an  analog- to-digital  con¬ 
verter  (ADC)  sami)liug  at  a  sub-Nyquist  rate.  The  type  of  RF  IF  spreading  employed  by  an  A2I 
receiver  includes  random  linear  projections  used  in  many  compressed  sensing  (CS)  applications  jl|, 
as  well  as  nonlinearities  described  in  subsequent  sections  of  this  document.  .A2I  receivers  have  ap¬ 
plication  to  electronic  intelligence  (FLINT)  and  signals  intelligence  (SIGINT)  systems  that  would 
lev'erage  the  additional  tlynamic  range  of  sub-Nyquist  sampling  ADCs  to  detect  weak  target  signa¬ 
tures.  For  example,  moving  from  a  1000  MS  PS  to  a  100  MSPS  ADC  using  present  .4  DC  technology 
would  increase  the  spur  free  dynamic  range  (SFDR)  by  roughly  30  dB. 

.421  receivers  will  also  benefit  intelligence,  surveillance  and  reconnaissance  (ISR)  sensor  net¬ 
works.  ISR  sensor  networks  that  pass  unprocessed  information  back  to  a  fusion  center  will  be 
j)erformauce-limite<l  both  by  power  consumption,  dynamic  range  and  available  communication  band¬ 
width.  The  capacity  of  an  ISR  receiver  to  digitize  a  wide  bandwidth  with  high  dynamic  range  and 
pass  the  j)0tentially  unprocessed  information  onto  a  fusion  center  under  tight  pow'er  and  communi¬ 
cation  bandwidth  constraints  is  a  significant  challenge.  Examples  of  such  sensor  networks  include 
the  Navy’s  Cooperating  Engagement  Capability,  which  is  forced  to  exchange  highly  processed  detec¬ 
tion  reports  as  opposed  to  unprocessed  multimotlality  sensor  data  due  to  communication  bandwidth 
immuration  |2|. 

Howxwer,  many  ISR  systems  operate  in  environments  populated  with  signals  that  are  not  so 
sjjarse  in  the  basis  in  which  they  are  compressible.  Examples  of  not-so-sparse  signals  inchule  those 
generated  by  many  modern  military  communications  systems  (e.g.,  M-ary  PSK/CPFSK,  Q.4M)  such 
as  MP-CDL  that  are  capable  of  supporting  data  rates  of  up  to  250  Mb/s,  and  commercial  802.11 
compliant  communications  devices  with  data  rates  in  excess  of  54  Mb/s.  Current  ISR  systems 
operating  in  such  environments  are  forced  to  implement  multiple  channel  narrowband  receivers 
that  are  expensive  in  terms  of  size,  weight  and  power,  or  use  high  sampling  rate  analog-to-digital 
converters  (.4DCs)  with  a  limited  dynamic  range. 

To  extenil  the  application  of  .421  to  SIGINT,  FLINT  and  ISR  platforms  operating  in  dense 
signaling  environments,  the  MIT  Lincoln  Laboratory/ G MR  Research  and  Technology  team  has  de¬ 
veloped  novel  signal  processing  algorithms  and  simulated  analog  IF  circuits  to  enable  next-generation 
ISR  sensor  netw’orks  with  high  information  rate  capacity,  high  dynamic  range  and  low  power  oper¬ 
ation.  This  report  will  detail  the  development,  performance  and  preliminary  mechanization  plans 
for  three  unique  progratn  thrusts;  Nonlinear  .Affine  processing  (NoLAfT),  Variable  Projection  and 
Unfolding  (VPU)  an<l  dynamical  systems.  In  the  following  sections,  we  will  demonstrate  both 
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VPU’s  and  NoLAff’s  unique  capability  for  identifying  and  reconstructing  not-so-sparse  signals  that 
are  digitized  with  a  sampling  rate  nearly  an  oriler  of  magnitude  lower  than  the  Nycjuist  rate.  In 
addition,  we  present  NoLAff’s  unique  approath  to  signal  recovery  vjsing  nonlinear  analog  circuitry 
and  nonlinear  digital  signal  processing,  and  a  novel  approach  to  an  analog  circuit  imjjlementation 
of  dynamical  systems  for  improving  the  signal  identification  capability  of  any  A2I  algorithm  beyond 
what  is  currently  possible  using  randomized  linear  projections  or  random  sampling.  The  nearly 
one  order  of  magnitude  reduction  in  sample  rate  below  the  Nyqnist  rate  using  the  A2I  techniques 
outlined  in  this  document  holds  the  potential  to  enable  the  deployment  of  single-channel  receivers 
with  high  dynamic  range  that  are  capable  of  operating  in  sensor  networks  aiul  passing  unprocessed 
information  requiring  nearly  one  order  of  magnitude  less  bandwidth  than  conventional  systems. 


NoLAff  and  VPU  Undersampling  NoLAff  processing,  specifically  NoLAff  hypothesis 
testing,  is  the  first  technique  explored  in  this  report  for  reduced  rate  sampling.  NoLAff  hyjwthesis 
testing  utilizes  an  analog  encoder  consisting  of  a  known  nonlinearity  into  which  the  received  signal 
is  injected  along  with  a  strong,  possibly  a  prion,  known  probe  signal.  One  can  view  the  addition  of 
the  probe  signal  as  an  affine  transformation  of  the  input  signal.  The  now-ambiguous  output  from 
the  sub-Nyquist  (say  M  times  undersampled)  ADC  is  decoded  to  reconstruct  the  full  spectrum  of 
the  received  signal  by  employing  hypothesis  testing.  The  nonlinear  artifacts  are  used  to  determine 
which  of  the  M  possible  ambiguous  copies  of  each  input  signal  is  the  correct  one.  Hence,  the 
nonlinear  artifacts  may  be  viewed  as  a  form  of  signal  diversity.  This  tecliniqtie  utilizes  a  form  of 
nonlinear  equalization  to  remove  the  nonlinear  artifacts  as  part  of  the  hypothesis  testing  and  the 
final  reconstruction.  Section  2  details  NonLinear  Affine  (NoLAff)  signal  processing,  identification 
and  recon.struction  performance,  computational  complexity,  and  a  first  cut  at  an  initial  hardware 
implementation. 

Like  Compressed  Sensing  (CS),  V’PU  leverages  the  property  that  a  small  number  of  random 
linear  projections  of  a  sparse  signal  contain  most  of  the  salient  information.  To  reconstruct  the 
original  signal,  VPU  is  used  to  find  a  sparse  signal  that  matches  the  random  linear  projections  of 
the  original  signal.  Simply  put,  V'PU  se<iuentially  shifts  and  coarsely  increments  a  column  pointer 
to  locate  the  basis  locations  that  the  signal  spans  and  then  orthogonally  projects  the  randomized 
and  downsampled  basis  onto  the  received  data  to  reconstruct  the  original  transmitted  signal.  If 
the  difference  between  the  received  signal  and  reconstructed  signal  is  small,  a  detection  is  declared 
and  those  basis  locations  are  used  when  search  continues,  .\fter  all  the  basis  locations  spanned 
by  candidate  signals  have  been  identified,  a  fine  search  is  conducted  where  basis  locations  are 
jointly  incremented  and  decremented  to  more  accurately  pinpoint  the  locations  in  the  basis  that  the 
signals  span.  \\%  have  found  that  in  dense  signaling  environments,  i.e.,  environments  in  which  the 
signals  occupy  more  than  15-20  percent  of  the  downsampled  basis  in  which  the  signal  has  a  sparse 
representation,  linear  programming  techniques  are  unable  to  effectively  reconstruct  the  original 
signal.  However,  unlike  linear  programming  techniques  used  in  compressed  sensing  applications  |l] 
for  identification  and  reconstruction,  VPU  is  capable  of  reconstructing  signals  that  occupy  up  to 
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nearly  100  percent  of  the  downsampled  Nyquist  band.  We  will  present  the  VPU  algorithm  in 
Section  3  and  compare  its  identification  and  ret;onstruc:tion  performance'  to  the  performanc’e  of 
linear  prograniining  used  in  compressed  sensing.  In  Section  5  we  will  derive  both  the  maximum 
likelihood  and  linear  programming  bound  which  we  will  use  to  support  the  empirical  results  in 
Section  3. 


Dynamical  Systems  In  applying  the  algorithmic  tct:hiiiques  of  compressed  sensing  to  RF 
sensors,  we  utilize  analog  signal  processing  that  incorporates  randomization  and  memory.  The 
processed  analog  data  are  sampled  at  a  rate  that  is  typically  well  below  Nyquist,  but  full-band 
reconstruction  of  signals  is  possible  given  sparse  signal  environments. 

Dynamical  systems  provide  a  natural  model  for  analog  signal  processing  with  the  reciiiisite 
memory  ami  randomness.  A  typical  dynamical  system  is  characterized  by  an  ordinary  diffen'ntial 
e<iuation  (ODE)  that  models  system  behavior.  Sampled  outputs  of  the  dynamical  system  can 
proviile  initial  conditions  for  the  ODE  that  allow  the  estimation  of  intersamjde  values,  supporting 
the  t:oucept  of  sampling  below  Nyquist  rates. 

When  the  dynamical  system  is  driven  by  an  unknown  input  signal,  sampled  values  can  be 
used  to  form  an  innovation  process  that  is  modeled  by  the  ODE  of  the  dynamical  system.  Given 
.sparseness  of  the  input  in  some  known  basis,  we  can  use  the  Stunpled  values  to  reconstruct  the  input 
signal.  The  precise  manner  in  which  we  can  achieve  this  recronstruction  is  discussed  below. 

The  tlynamica!  systems  of  practical  interest  are  capable  of  hamlling  wideband  microwave 
signals.  .4n  important  aspect  of  such  a  dynamical  system  can  be  signal  delay  in  feedback  loops. 
We  consider  a  specific  dynamical  system  with  feedback  delay  and  form  aii  ODE  model  of  what  is 
inherently  modeled  by  a  tlelay  tlifferential  eejuation  (DDE).  This  ODE  model  is  crucial  since  discrete 
sampling  does  not  make  sense  in  the  context  of  DDE, 

Most  of  the  work  presented  here  establishes  an  infrastructure  for  system  modeling  and  signal 
recovery.  Anecdotal  results  are  shown  for  tire  successful  recovery  of  undersampled  wideband  luilsed 
wavfrforms.  Parameters  of  the  dynamical  systems  have  not  been  optimized  in  any  way. 


Performance  Bounds:  Compressed  Sensing  Much  of  the  compressed  sensing  litera¬ 
ture  is  focused  on  practical,  iterative  techniques  that  avoid  the  combinatorial  searches  inherent  in 
straightforward  approaches  to  sparse  signal  recovery.  The  typical  formulations  attribute  a  single 
response  vector  to  each  signal.  Signal  recovery  amounts  to  finding  tfiese  vectors  and  the  correspond¬ 
ing  signal  aniplitmles  that  explain  the  observed  data.  The  sparsity  reciuire<l  for  perfec-t  recovery 
in  noiseless  environments  or  stable  recovery  in  noise  is  typically  bounded  asymptotically  as  the 
number  f)f  response  vectors  increases.  In  this  limit,  each  signal  is  still  a  single  response  vector.  This 
type  of  limiting  performance  does  not  take  advantage  of  a  priori  information  about  signals  that  can 
potentially  improve  signal  recovery. 
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Consider,  for  example,  a  fixed  collection  of  finite  bandwitltli  signals  with  unknown  center  fre¬ 
quencies.  If  the  receiver  bandwidth  is  also  fixed,  a  limiting  regime  can  involve  increasing  observation 
times,  resulting  in  an  increase  in  the  effective  number  of  samples  for  each  signal.  Abstractly,  the 
number  of  response  vectors  associated  with  each  signal  grow'S  in  a  structured  inaiiiier.  This  structure 
and  the  additional  sample  .support  can  be  used  to  improve  recovery.  Note  that  recovery  remains 
prima  facie  a  combinatorial  problem. 

In  Section  5.3  we  introduce  a  more  general  formulation  of  basis  pursuit  tliat  can  handle  the 
structured  signals  described  above.  For  intuition,  one  should  keep  in  mind  the  example  involving 
finite  bandwidth  signals  with  unknown!  center  frequencies  and  sparse  total  occupancy.  We  present 
lower  bounds  on  the  fraction  of  the  band  that  can  be  perfectly  reconstructed  in  the  absence  of  noise. 
We  will  show'  that  this  bound  is  substantially  larger  than  one  commonly  used  for  basis  pursuit.  We 
also  propose  a  mixed  norm  functional  to  handle  stable  recovery  in  noise.  An  iterative  algorithm  to 
minimize  this  functional  is  also  proposed. 

Performance  Bounds:  Covariance  Estimation  We  use  a  statistical  signal  model  in 
order  to  assess  the  penalty  associated  with  undersampling  wideband  data.  Specifically,  w'e  assume 
that  signals  can  be  described  by  unknown  covariances  in  a  knowm  signal  basis.  Assuming  that  the 
covariance  is  in  some  sense  sparse,  w'e  can  evaluate  the  loss  in  performance  due  to  undersamijling. 
The  calculations  are  based  on  standard  linear  randomization  and  imdersami)ling  inotlels. 

Given  sufficiently  sparse  covariances,  a  Cranier-Rao  bound  show's  that  the  loss  in  performance 
associated  with  undersampling  grows  as  the  septare  of  the  uiidersarapling  factor.  The  Cramer- 
Rao  bound  does  not  address  all  loss  mechanisms  since  it  characterizes  loc’al  (near  the  true  value) 
behavior  of  unbiased  estimators.  Good  estimators  typically  approach  the  l)Ound  at  a  .sufficiently 
high  signal-to-!ioise  ratio  (SNR),  demarcating  the  boundary  betw'ceii  global  aiul  local  behavior  of 
the  estimator. 

The  results  shown  rely  only  on  Cramer-Rao  bounds  for  estimates  of  covariance  parameters. 
Sparsity  is  used  to  evaluate  the  hounds  but  is  not  utilized  by  the  receiver  to  improve  performance. 
In  some  sense,  sparsity  is  a  form  of  a  priori  know'ledge  and  tlius  could  be  incorporated  into  random 
parameter  Cramer-Rao  bounds  to  improve  performance.  For  exain])le,  the  mixed  La,  L\  norm 
objective  functions  that  are  often  used  for  signal  recovery  can  be  interpreted  in  terms  of  Laplacian 
priors  |3|  on  the  signal  amplitudes.  R,esults  of  this  nature  are  not  presented  in  this  report. 
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2.  NONLINEAR  AND  AFFINE  SIGNAL  PROCESSING 


NonliHcarities  in  many  receiver  and  signal  processing  applications  are  viewed  as  eitlier  a  nui¬ 
sance  to  t)e  avoided  or  overcome.  By  engineering  systems  to  iiave  almost  perfect  linear  charac¬ 
teristics,  one  can  reduce  nonlinear  effects  albeit  at  a  price  that  usually  involves  compromise  in 
j)erformance  otherwise  (e.g.,  backing  off  from  saturation  in  amplifiers  and  consequently  paying  a 
price  in  linear  efficiency  or  SNR).  Alternatively,  one  can  live  with  some  nonlinearities  in  the  system 
and  consecjuently  reduce  the  nonlinear  signal  artifacts  (e.g.,  intermodulatioii  products  and  harmon¬ 
ics)  after  the  fact  (e.g.,  using  digital  signal  processing  equalization).  In  this  case  the  price  paid  is 
one  of  computational  complexity  and  the  power  consumption  associated  with  it. 

In  this  section  we  consider  the  possibility  of  exploiting  nonlinearities  and  nonlinear  signal 
artifacts  to  the  benefit  of  the  overall  system.  Rather  than  compromising  performance  or  actively 
getting  ritl  of  distortions  w'e  use  nonlinear  signal  artifacts  as  a  source  of  additional  information  on 
the  signals  of  interest.  In  particular  w'e  introduce  the  concept  of  NonLinear  and  .Affine  (NoL.Aff)  j4| 
signal  proce'ssing,  w'hich  aims  to  exploit  nonlinear  interactions  betw'cen  various  signals  in  the  si)a<T 
of  interest.  Here  we  have  a  strong  sigiral,  possibly  oiir  own  probe  signal,  which  dominates  other 
signals  in  the  space  of  interest.  The  weaker  signals  may  be  viewed  as  small  perturbations  of  signals 
existing  on  a  differentiable  manifold  described  by  the  nonlinear  system.  Here  the  strong  prolie 
controls  the  neighborhood  on  that  manifold  where  the  signals  exist. 

This  form  of  nonlinear  function  -  NoL.4ff  -  is  nearly  linear  in  the  small  signal  with  all  the  ad¬ 
vantages  that  creates  w'hile  having  some  of  the  benefits  of  a  nonlinear  system  (e.g.,  signal  diversity). 
The  choice  of  probe  signal  gives  the  system  design  degrees  of  freedom,  which  may  be  exploited  and 
easily  controlled. 

Figure  1  tlepicts  a  notional  use  of  NoLAff  where  a  weak  signal  of  interest  (shown  here  in  blue) 
enters  the  system  along  with  a  strong  signal  (showm  in  red)  w-hich  may  exist  in  the  eiivir<mmeiit  or 
which  may  be  a  probe  signal  injected  on  j)urpose.  Tiie  output  is  such  that  aside  from  the  expected 
linear  signal  components  we  have  higher  dimensional  nonlinear  artifacts  which  contain  copies  of  tiie 
original  signals  as  well  as  cross  products  (shown  in  purple).  Tliese  provide  a  form  of  signal  diversity. 
While  the  strong  signal  is  easy  to  detect  and  process,  the  weaker  one  now  has  multiple  copies,  which 
processed  togetiier  may  be  used  to  increase  its  apparent  dynamic  range. 

In  the  context  of  analog- to-iiiforniation  we  use  NoL.4ff  processing  as  an  approach  to  reduced 
rate  .sampling  of  signals.  Here  we  consider  the  case  where  the  signal  of  interest  is  known  to  be  sparse 
in  the  sen.se  that  its  rate  of  innovations  is  significantly  lower  than  the  Nyquist  or  Shannon  sampling 
rate.  In  particidar  W'e  assume  that  there  is  a  linear  decomposition  of  the  signal  of  interest  such  that 
very  few  of  the  basis  vectors  used  to  represent  it  are  nonzero.  A  somewhat  deeper  discussion  of 
si)arseness  is  given  elsewhere  iji  tins  report;  hence,  we  concentrate  here  on  a  particular  example  of 
signals  which  have  a  sparse  but  a  priori  unknowm  representation  in  the  fretiuency  domain. 
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Figure  1:  Signal  (blue)  and  strong  probe  signal  (red)  pass  through  receixier  with  noniinearities. 
Output  contains  linear  components  and  nonlinear  components  (here  only  second  order  shown). 
This  additional  infoTmation  on  the  signal  from  highei'  order  componenls  (purple)  allows  xis  to 
extract  more  informatiori  on  the  signal  of  intef'est  than  linear  processing  would. 
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L<'t’s  assunip  hero  that  the  sparse  input  signal  j;(/)  (where  /  represents  the  fretiuency  varia!)le) 
is  smh  that  x{f)  =  0  for  all  /  not  in  the  Ijaiicl  [0,  B]  with  the  obvious  extensions  to  any  other 
passhand.  While  the  frequency  support  of  a?  is  sparse  during  any  time  interval  of  interest,  do  not 
know  the  exact  sui)port.  Hence  classically  we  either  have  to  sample  at  a  rate  of  at  least  2D  or  risk 
having  unrecoverable  aliasing  distortions;  or  bandpass  filter  the  input  signal  to  a  smaller  support 
baml  and  sample  at  the  commensurate  rate  and  potentially  lose  all  information  outside  the  band 
of  the  filter.  We  claim  that  NoLAff  processing  (both  analog  and  digital)  will  allow  us  to  sample 
at  a  rate  significantly  lower  than  2B  without  losing  any  information.  While  sampling  at  2D  seems 
to  be  a  good  solution,  we  remember  that  there  are  practical  limitations  to  sampling.  For  some 
l)andwidths,  there  simply  are  no  COTS  ADCs  available  and  even  when  there  are  ADCs,  we  recall 
that  high-rate  .4DCs  have  significantly  lower  dynamic  range  than  the  slower  counterparts  and  they 
con.sume  more  power  and  cost  more  typically. 

A  Nyquist  rate  ADC  sampling  receiver  is  depicted  notionally  in  Figure  2.  Here  we  notice  the 
limitetl  SNR  of  the  signal  after  the  ADC.  While  no  information  was  lost  in  the  sense  of  filtering  or 
aliasing,  some  signals  may  be  lower  than  the  noise  and  hence  unrecoverable.  On  the  other  hand, 
(consider  the  NoL.Aff  .ADC  saniijling  scheme  depicted  in  Figure  3  where  a  low  rate  ADC  is  used, 
thereby  creating  aliasing  <listortion  but  having  significantly  higher  dynamic  range.  Here  NoL.Aff 
[jrocessing  is  used  to  rejuove  tlie  aliasing  distortion  while  retaining  the  tlynamic  range  of  the  low 
sample  rate  .ADC.  In  this  section  we  describe  one  actual  method  of  using  NoL.Aff  processing  to  do 
just  that. 

With  the  remaiiulcr  of  Section  2,  we  present  NoLAff  signal  processing  in  the  context  of  .A21.  In 
Settion  2.1,  we  introduce  sub-Nyquist  sampling  and  bounds  on  unambiguous  signal  reconstruction. 
One  method  for  undersampling  and  reconstructing  a  sparse  signal,  NoLAff  hypothesis  testing,  is 
detailed  in  Section  2.2,  and  a  discussion  of  its  computational  complexity  follows  in  Section  2.3. 
Section  2.4  provides  a  brief  explanation  of  how  NoLAff  undersampling  relates  to  another  .A2I  tech- 
ni^lue,  Cotnpre.ssed  Sensing.  In  Section  2.5,  we  present  a  simulated  hardware  implementation  of 
NoL.Aff  Hypothesis  Testing,  and  conclude  by  detailing  the  simulations  performed  and  discussing 
their  results  in  Section  2.6. 


2.1  NOLAFF  SAMPLING  FOR  “SPARSE”  SIGNALS 

In  the  context  of  sub-Nyquist  sajnpling  of  signals  with  a  limited  frequency  support  or  more 
generally  a  limited  representation  in  some  basis  D,  we  wish  to  explore  the  bounds  on  tmamliiguous 
signal  reconstruction  or  extrapolation. 

Let  X  be  a  signal  of  length  m.  We  assume  that  all  signals  of  interest  (with  the  exception  of 
noise)  belong  to  a  fixed  subspace.  Denote  by  D  an  m  x  np  matrix  whose  columns  form  a  basis 
s|janning  tlie  fixed  subspace.  If  only  nx  <  components  of  the  measured  data  vector  y  =  Dx 
are  observed,  we  wish  to  identify  x  unambiguously  from  y.  Let  Pr  denote  the  projector  onto  the 


Figure.  2:  Standard  Nyquist  rate  sampling.  Notice  the  limited  SNR  of  the  signal  after  the  ADC. 


External 


Figure  3:  Suh-Nyquist  NoLAS  sampling.  A  low-rnte  ADC  is  used,  thereby  creating  aliasmg 

distortion  but  hauing  sigiiificantly  higher  dynamic  range.  Here  NoLA  ff  processing  is  used  to  remove 
the  aliasing  distortion  while  retaining  the  dynamic  range  of  the  low  sample  rate  ADC. 
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Figure  4:  Diagram  of  the  operation  of  a  general  NoLAff  undersampUng  system. 


stibspace  spanned  by  the  77 t  observed  components  of  y.  This  subspace  is  spanned  by  the  columns 
of  an  n  x  tit  matrix  T.  If  another  nonzero  signal  x  satisfies  Pry  =  PrDx,  then  D(x  —  x)  is  a 
nonzero  element  of  the  ortho-complement  of  Pp. 

Let  T  and  2?  denote  tlie  subspaces  spanned  by  the  f:ohimns  of  T  and  D  respectively.  Then 
nV  ^  <t>  where  •‘^represents  the  ortho-complement  operator.  Now- 

dim n  P  >  max(diin  -I-  dim P  —  dim 77, ()))  =  max(77D  “  0)  (1) 

with  equality  typically.  That  is,  we  would  expect  T^DP  =  0  whenever  tit  >  77d-  Hence,  when 
the  sample  support  is  greater  or  equal  to  the  received  signal’s  subspat*  dimension,  we  would  expert 
unambiguous  reconstruction  typically.  Specifically  for  bandlimited  signals,  this  would  achieve  the 
Nytjuist  rate.  The  results  in  Section  2.6  indeed  show  that  signals  that  occupy  a  subspace  with 
dimensions  commensurate  with  the  sampling  subspace  dimension  can  be  reconstructed  uniquely. 

2.2  UNDERSAMPLING  USING  NOLAFF 

Traxlitioually,  sampling  requires  a  rate  of  at  least  tw'ice  the  frequency  support  of  the  signal. 
However,  in  many  circumstances,  the  desired  signal  is  sparsely  located  within  a  large  bandwidth 
with  some  n  priori  unknown  support.  Despite  occupying  only  a  small  portion  of  the  full  bandwidth, 
traditional  sampling  requires  such  a  signal  to  be  sampled  at  the  full  Nyquist  frequency.  NoL.4fr 
undersanipling  provides  a  method  for  reducing  the  required  sampling  rate,  closer  to  the  infornuition 
rate  for  a  sparse  signal. 

The  NoLAff  undersampling  system  described  here  consists  of  tw*  major  parts,  showui  in  Figure 
4.  First  is  an  analog  system  frojit-end,  wiiich  encodes  signal  information  through  a  nonlinearity 
and  samples  the  resulting  signal  below'  the  traditional  Nyquist  rate.  The  second  part  processes 
the  undersanipled  signal  via  hypothesis  testing,  decoding  the  signal  information  and  placing  the 
received  signal  into  the  proper  location  within  the  full  bandwidth.  Both  parts  are  descriljed  l)elow. 

NoLAff  Encoding  Undersampling  a  signal  results  in  aliasing.  If  a  sparse  signal  is  knowm 
to  have  been  undersampled,  it  will  contain  an  image  of  the  original  signal  which  was  folded  into 
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Figure  5:  System  diagram  for  a  general  NoLA ff  analog  encoder  front-end. 

the  smaller  umiersampled  spectrum.  In  order  to  reconstruct  the  original  signal  at  its  full  rate,  the 
original  frequency  location  of  the  aliased  image  must  be  determined.  Without  extra  information  from 
the  signal  and  the  system  it  passed  through,  it  is  impossible  to  do  this.  However,  by  sf:ireading  signal 
information  into  more  frequency  locations  and  filtering  the  results,  NoLAff  allows  an  undersampled 
sparse  signal  to  be  reconstructed. 

Signal  spreading  creates  extra  copies  of  the  original  signal  at  deterministic  freciiiency  lot'ations 
in  the  full  spectnim.  This  alone  is  not  enough,  as  these  extra  copies  also  have  ambiguous  freciuency 
information  after  undersampling.  These  extra  copies  are  not  ambiguous  after  they  have  been  filtered, 
as  the  filtering  process  imparts  an  alteration  to  the  images  wdiich  is  unique  based  on  frequency 
location  in  the  full  spectrum.  These  adjustments  are  dotenninistk,  and  are  later  used  by  hvpotliesis 
testing  to  choose  the  most  likely  origin  of  the  signal. 

The  NoLAff  encoder  is  an  analog  system  which  spreads  a  sparse  input  signal  and  filters  the 
result  to  provide  information  for  reassembling  the  signal  after  undersampliiig.  Figure  5  shows  a 
general  NoLAff  encoding  system.  This  achieves  both  essential  aspects  for  reconstruction  of  an 
undersampled  signal  using  NoLAff  encoding  which  are  explained  below. 


NoLAff  Information  Spreading  A  general  NoLAff  encoder  is  shown  in  Figure  5,  This 
function  is  nonlinear  and  can  be  represented  by 

y  =  f{x).  (2) 

In  general,  this  function  adds  a  strong  probe  signal  to  the  input,  passes  this  result  through  a 
polynomial  filter,  then  removes  the  probe  and  the  harmonic  distortions  resulting  from  the  i)robe, 
leaving  the  input  signal,  intermodulation  products  hetwx^en  the  probe  and  input,  as  w^ell  as  higher- 
order  terms.  All  information  regarding  the  probe  characteristics  is  assumed  to  be  known  a  priori. 
The  NoLAff  nonlinear  function  can  be  expressed  as 

f(x)  =  g(x  +  p)  -  g(p),  (3) 
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where  p  represents  the  probe  signal  and  the  function  g(-)  implements  a  general  nonlinearity  repre¬ 
sented  by  a  polynomial  filter 


i=l 

W’ithout  loss  of  generality,  we  assume  the  nonlinearity  to  be  inemoryless  except  for  the  filter.  The 
operations  for  element-wise  multiplication  and  exponentiation  are  denoted  by  and  re¬ 

spectively.  In  this  derivation,  we  require 


llp||»l|x|l. 


such  that 


f(x)  =  g(x  +  p)~g(p) 

%  *  p 

k=\ 

DC 

=  X.  ♦  (5) 

*r=l 


Therefore,  NoLAff  is  linear  with  respect  to  the  input  x.  This  result  may  be  interpreted  as  a 
modulation  of  the  input  signal  by  k  —  1  copies  of  the  probe,  where  k  is  the  order  of  the  polynomial 
term.  When,  for  example,  the  probe  is  a  single  tone,  each  modulation  produces  a  copy  of  the  input 
signal  at  a  different  frequency  location,  and  the  addition  of  several  polynomial  orders  prodiu^es 
many  copies  of  the  original  signal  spread  throughout  the  frequency  spectrum.  As  a  determini.stic 
operation,  these  modulations  can  be  repeated  exactly  during  NoLAff  decoding. 

.\n  illustrative  example  is  the  third-order  system 


.V  «  X  -t-  a.jx.  ♦  p 


(6) 
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For  simjjlicity,  we  let  x  and  p  be  complex  exponential  vectors,  with  x  =  cj 


+  c.c:.  and 


e-jw, 


P  =  Cp 


e-J^P 


+  C.C.  w’here  jcp|  »  |ci|.  From  equation  6,  we  have 


1' 

1 _ 

g-j(2sJp+u/i) 

y  =  (1  +  2n3f^)  cj 

+  «3CKp 

p— ) 

g-j(2u;p-wi) 


£>“  j2(2u*p— Wi ) 


+  C.C. 


(7) 


Clearly,  the  output  frequency  support  is  greater  than  the  input  frequency  support* 


NoLAff  Signal  Filtering  Spreading  of  the  signal,  by  itself,  prior  to  undersampling  is  not 
enough  to  allow  for  the  original  signal  to  be  reconstructed.  To  differentiate  between  the  different 
aliased  images,  or  the  frequency  zones  from  where  they  came,  the  nonlinear  signal  is  filtered  prior 
to  imdersampling*  This  filtering  encodes  the  intennodulation  products  by  altering  amplitude  and 
phase  values  of  the  distortions  based  on  their  frequency  location  in  the  full  spectrum. 


Two  examples  of  filters  used  for  encoding  are  given  in  Figure  6.  Neither  filter  is  ideal,  but 
both  work  well  and  provide  an  intuitive  feel  for  how  the  filtering  encodes  frequency  information. 
Filter  A  is  a  digital  high-pass  filter  defined  by  1  —  ^  ;  filter  B  is  an  albpass  delay  filter 

given  by  h  = 


0  1 


phase  changes  at  eac 


Filter  A  encodes  a  signal  by  causing  a  unique  combination  of  amplitude  aiid 
frequency  location  wdthin  the  bandwidth.  Encoding  based  on  phase  alone 
is  demonstrated  by  Filter  B.  The  all-pass  nature  of  Filter  B  gives  a  constant  magnitude  response 
across  the  spectrum.  This  is  advantageous  in  low  SNR  cases. 


The  all-pass  filter  also  provides  a  convenient  demonstration  of  the  frequency  encoding  that 
takes  place  with  NoLAff.  The  phase  response  for  Filter  B  is  re-examined  in  Figure  7,  wludi  shows 
the  differences  that  occur  in  potentially  ambiguous  undersainpled  tones.  The  diagram  assumes  the 
signals  were  vmdersarnpled  by  a  factor  of  4,  leading  to  four  tones  from  the  full  spectrum  that  woulti 
alias  to  the  same  location  in  the  low-rate  spectrum.  These  are  shown  in  green.  Due  to  the  phase 
delay  imposed  by  the  all-pass  filter,  the  four  tones  from  the  full  spectrum  each  receive  a  unique  phase 
change.  NoLAff  decoding  exploits  these  differences,  which  are  applied  to  the  nonlinear  distortions, 
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Magnitude  Response  for  Filler  A  Phase  Response  for  Filter  A 


Magnitude  Response  for  Fifier  B  Phase  Response  for  Filter  B 


6:  Magnitude,  and  phase  responses  for  two  example  filters.  Filter  A  provides  encoding  of 
frequency  location  based  on  both  arnpHtude  (upper  left)  and  phase  (upper  r\ght)  changes.  Filter  B. 
an  all-pass  filter ^  has  a  constant  amplitude  gain  (lower  leftjf  but  phase  delay  (lower  tight)  prwides 
the  frequency  encoding. 
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Filter  B:  Frequency  Encoding  via  Pha^e  Delay 


Figure  7:  An  all-pass  filter  can  be  used  to  encode  the  frequency  lomtion  of  aliased  images.  This 
example  has  an  undersample  ratio  of  4f  lending  to  four  frequency  zones  from  where  the  signal 
(green)  could  originate.  The  undersampled  signal  in  this  case  would  measure:  a  center  frequency  of 
0,2,  normalized  to  the  fulTrate  bandwidth.  However,  the  measured  phase  dfday  would  be  diffei'cnt 
for  each  image,  -36°, -54°, -126°,  or  -144°,  corresponding  to  the  full-rate  signals  centered  at  the 
normalized  frequencies  0,2,  0.2,  0,7  or  0,8,  respectively. 


to  tiecide  which  of  the  signal  options  from  the  full  spectrum  most  closely  resembles  the  ineasvirefh 
undersampled  signal. 


Hypothesis  Testing  After  a  sparse  signal  is  encoded,  it  is  sampled  at  a  sub-Nyqiiist  rate. 
The  full  encoding  and  undersampling  process  is  depicted  in  Figure  8.  Tlie  result  is  a  measured  signal 
with  aliased  images  and  encoded  noiilinearities.  The  NoLAff  processor  reconstructs  the  original 
sparse  input  signal  via  hypothesis  testing,  depicted  in  Figure  9.  Aliasing  is  viewed  as  splitting  the 
full- rate  bandwidth  into  equal-length  frequency  zones,  which  fold  on  top  of  each  other  during  the 
undersanipling  process.  Each  hypothesis  is  passed  through  a  copy  of  the  nonlinear  encoding  system, 
and  a  Bayesian  decision  yields  the  ML  unfolded  signal. 

Hypothesis  testing  requires  several  steps  to  arrive  at  the  most  likely  input  signal  Each  hy¬ 
pothesis  signal  is  generated  from  the  measured  signal  parameters  which  are  transferred  into  the 
appropriate  full-rate  frequency  zone.  Then,  each  hypothesis  signal  is  individually  passed  though  the 
encoding  system:  the  probe  signal  is  added,  it  is  passed  through  the  cubic  function  and  filtered,  and 
the  probe  is  removed.  Each  hypothesis  is  then  undersampled,  resulting  in  the  hypothesis  out]>ut 
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4 


INPUT 

▼ 


NoLAff  Encoding 


Undersampling 


FigujT.  8:  NoLAff  undermmpiing  spreads  and  encodes  a  sparse  input  signal  Although  under- 
sanipling  creates  aliased  images  which  fold  on  top  of  each  otheTf  the  encoding  process  allows  the 
origmal  signal  to  be  reconstructed. 
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Figure  9:  The  decoding  process  for  hypothesis  testing  requires  seAmrai  steps.  First,  the  strongest 
portion  of  the  mensured  signal  is  uolated  and  eqttalized  by  the  iterative  NLEQ  proces.s.  This  create.^ 
a  set  of  hypothe.sis  input  .signah  (blue),  each  of  which  is  processed  through  the  NoLAff  encoder, 
with  the  results  compared  against  the  mea,sured  .signal.  The  closest  matching  hypothesis  is  taken 
to  be  the  output  and  the  most  likely  choice  for  the  input  prior  to  ejicodmg. 
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Intermod! 


1 

NoLAff  j 
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Input  Output 


FiffiifT  10:  The  NoL A ff  process  generates  a  strong  interrnod  which  falls  on  top  of  the  input  signaL 
Iterative  NLEQ  is  used  to  find  the  input  signal  amplitude. 

signals*  A  wirmliig  liypothesis  is  chosen  by  the  ML  criterion.  The  ML  is  found  by  calculating  tlie 
correlatitHi  metric  |5| 

C^(yi  2y  ■  , 

where  y  is  the  measured  signal  and  is  the  hypothesis  output  signal,  and  then  finding 
k  =  argmax(C(y,z„0), 

ni 

giving  the  k'^'  hyjHUhesis  as  the  niost  likely  dioice  of  the  input  signal. 


Iterative  NLEQ  In  the  process  of  generating  the  NoLAff  hypotliesis  signals,  it  is  ne<'essary 
to  perform  a  nonlinear  equalization  (NLEQ)  of  the  signal.  The  strongest  frequency  bins  in  tlie 
niea-siiretl  signal  spectrum  are  translated  into  each  frequency  zone  in  the  full  spectrum  as  the  first 
step  in  creating  the  hypotheses.  However,  the  output  of  the  NoLAff  encoder  has  pas.sed  through  the 
nonlinearity,  while  the  liypotbe.sis  testing  system  requires  an  input  signal  prior  to  passing  through 
the  uouliuearity.  This  is  important  because  any  odd  power  nonlinearity  will  produce  a  strong 
inter  modulation  term  located  at  the  same  frequencies  as  the  input  signal.  Since  this  distortion  term 
falls  on  toj)  of  the  information  needed  to  geiierate  the  hypothesis  signals,  an  iterative  NLEQ  process 
is  used  to  approximate  the  needed  information, 

.An  example  of  this  can  be  seen  with  the  cubic  nonlinearity  and  linear  passthrough  from 
etiuation  (7).  It  is  clear  tliat  the  cubic  nonlinearity  generates  a  tone  at  the  same  frequency  Wi  a.s  the 
linear  ]>a.ssthrough  tone,  and  the  result  is  a  tone  constructed  from  the  combination  of  the  two.  This 
NoLAff  operation  is  depicted  in  Figure  10,  where  the  green  signal  represents  the  linear  passthrough 
and  the  re<l  signals  are  the  nonlinear  distortions.  NoLAff  needs  an  accurate  measure  of  the  green 
signal,  and  iterative  NLEQ  is  used  to  separate  it  from  the  red  signal. 
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y  =  NL( input  +  probe)  -  NL( probe) 

Y  =  FFT(y) 

loop 

Zi(n+alias)  =  Y(n) 

Zi=  iFFT(Zi) 

2i=  NL(zi+  probe)  -  NL(probe) 
y=  y  -  (2i-  Zi) 

Y  =  FFT(y) 
until  Zi  stabilizes 
repeat  for  all  i 


Figure  11:  Pseudocode  for  the  iterative  NLEQ  process.  Iterative  NLEQ  adjusts  a  hypothesis 
signal  so  that  it  will  produce  the  closest  estimate  to  the  received  signal  after  passing  through  the 
nonlinearity. 

Pseudocode  for  the  iterative  NLEQ  process  is  shown  in  Figure  11.  The  process  begins  by 
measuring  the  output  signal  y  and  generating  the  initial  set  of  hypothesis  signals  Zi.  This  was  done 
by  taking  the  DFT  of  y,  translating  the  frequency  bins  relating  to  the  signal  to  their  respective  alias 
locations  in  the  full  spectrum,  then  performing  an  inverse  FFT.  Each  hypothesis  is  then  added  to  a 
copy  of  the  probe,  passed  through  the  nonlinear  system.  The  probe  and  its  harmonics  are  sul)tracted 
off.  The  hypothesis  signal  prior  to  the  nonlinear  system  is  subtracted  from  the  hypothesis  signal 
after  the  nonlinear  system,  in  order  to  isolate  the  intermodulation  distortions.  Those  distortions  are 
then  subtracted  from  y,  which  are  used  to  generate  the  next  iteration  of  zi.  The  process  iterates 
until  a  stable  hypothesis  signal  Zi  is  found.  The  process  is  repeated  for  each  hypothesis  signal. 

We  note  two  points  regarding  iterative  NLEQ.  First,  this  algorithm  requires  the  nonlinear 
distortions  to  be  weaker  than  the  input  signal  in  order  to  be  stable.  This  counters  the  need  for  strong 
nonlinearities,  which  overcomes  noise  better  and  thus  reduce  the  chance  of  error  in  the  hypothesis 
decision.  A  good  balance  was  found  to  be  a  10  dB  power  reduction  from  the  linear  passthrough 
signal  to  the  dominant  3rd-order  interniod.  Second,  with  low  noise  and  simple  inputs,  only  one 
hypothesis  should  equalize  well.  The  wrong  hypotheses  will  generate  new  nonlinear  distortions 
while  attempting  to  equalize  the  ones  initially  present.  Despite  this,  all  hypotheses  reach  a  stable 
signal  when  the  nonlinearity  is  of  a  proper  strength. 


2.3  COMPUTATIONAL  COMPLEXITY 

Analyzing  the  computational  complexity  of  NoLAff  decoding  is  essential  for  determining  the 
feasibility  of  implementing  hypothesis  testing  in  a  real  system.  In  this  section,  an  explanation  and 
a  rough  estimate  of  the  processing  requirements  for  NoLAff  hypothesis  testing  are  described. 
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The  conij)lexity  of  the  full  NoLAff  hypothesis  testing  algorithm  builds  from  several  sutipro- 
cesses  and  is  a  function  of  several  variables.  The  NoLAff  variables  affecting  complexity  are  the  laxler- 
sample  rate  m,  ranging  from  1  (full  rate)  to  8  and  beyond,  and  the  bandwidth  of  the  signal,  indicated 
by  the  number  of  DFT  frequency  bins  b  the  signal  occupies.  The  number  of  frequency  bins,  6,  relates 
to  tlie  signal  biuidwitlth  as  a  percentage  of  the  full-rate  bandwidth  by  b  =  ceil  {%BW  x  N/2).  The 
primary  operations  in  NoLAff  are  the  iterative  NLEQ  (performed  once),  complex  multiplication 
for  creating  hypothesis  signals  (repeated  mb  times),  low-rate  norm  and  maximum  likelihood  (ML) 
calculations  (repeated  mb  times),  and  the  nonlinearity,  (repeated  m  times).  Since  the  complex 
multiplications,  needed  for  generating  the  hypothesis  signals,  require  four  multiplications  per  data 
point,  this  adds  irnbN  operations  to  the  complexity.  The  low-rate  norm  and  ML  calculations  eacli 
require  ^  multiplications  and  are  repeated  three  times  for  each  hypothesis,  giving  ZN  operations. 
Combining  this  information  gives  a  comi)lexity  of 

4m bN  +  ZN  -f-  iterative  NLEQ  -I-  m  x  nonlinearity 

per  sample.  The  computational  complexity  of  the  iterative  NLEQ  and  nonlinearity  are  discu.ssed 
below. 

The  complexity  of  the  iterative  NLEQ  subprocess  is  ba.sed  on  one  additional  factor;  the  number 
of  iterations  /  performetl  to  equalize  the  uonliuearities.  The  number  of  iterations  typically  would 
range  from  5  to  15.  The  calculation  for  the  complexity  of  the  iterative  NLEQ  operation  rejjuires 
a  low-rate  (^)  FFT  (performed  ml  times),  a  complex  multiplication  for  creating  intermtHliary 
hypothesis  signals  (repeated  mib  times),  and  the  nonlinearity  (repeated  ml  times).  Combining 
this  information  .shows  that  iterative  NLEQ  requires  approximately  (L  -f-  4f> -I- 2 -H  ^ 
calculations  to  complete. 

The  nonlinearity  function  block  must  be  called  multiple  times  during  hypothesis  testing.  In 
a  typical  re(;eiver  NLEQ  system,  the  nonlinearity  calculation  could  require  sixteen  or  more  opera¬ 
tions  each  of  filters  and  multiplications  for  each  sample  |6, 7|.  However,  the  nonlinearity  in  these 
sinmlations  is  assumed  to  be  simple,  consisting  of  only  a  .simple  cubic  term,  requiring  two  multiplies 
per  data  sample,  and  one  filter  operation  per  sample.  The  length  of  the  filter  L  becomes  an  addi¬ 
tional  variable  in  the  computational  complexity.  The  nonlinearity  results  in  2N  or  {L-\-2)N. 

calculations  to  complete. 

The  complexity  of  the  full  NoLAff  hypothesis  testing  becomes 


( 


m  m 


^IN  +  m{L  +  2)N, 


which  siniplifie.s  to 


N  I\oyr—  +  ,n{I+l){L  +  4b  +  2) 
m 
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To  provide  a  better  understanding  of  how  each  of  the  factors  affects  the  computational  com¬ 
plexity,  a  set  of  typical  values  is  applied,  holding  all  but  one  factor  to  a  reasonable  value.  Typical 
values  used  were  /  =  10  for  NLEQ  iterations,  L  =  8  for  the  filter  length,  w  =  8  for  the  undersami)le 
ratio,  and  h  =  5  for  a  sparse,  1%  of  full  bandwidth  signal  when  N  ^  lOOO.  The  computational 
complexity  of  hypothesis  testing  per  input  sample,  as  a  function  of  iterations,  becomes  f\{I)  «  c\I, 
where  ci  is  a  constant.  Likewise,  the  complexity  is  approximately  linear  with  respect  to  undersample 
rate,  /2(m)  «  C2m,  and  with  respect  to  signal  bandwidth,  fs{b)  %  csb  -h  (4, 

2.4  CONNECTING  NOLAFF  TO  CS 

It  is  instructive  to  compare  the  NoLAff  approach  to  other  A-to-I  techniques.  In  particular,  we 
compare  the  NoLAff  encoder  to  the  randomization-type  encoders.  While  the  mechanics  of  decoding 
are  quite  different,  there  are  commonalities  in  the  encoding.  Much  like  randomization,  nonlinear 
and  affine  transformations  spread  sparse  signals  over  much  of  the  dictionary  representation  in  which 
they  are  sparse. 

Let 

X  =  TO,  (8) 

be  an  input  signal  where  T  is  a  dictionary  whose  columns  span  the  signal  space  of  interest  and  the 
vector  0  describes  the  linear  combination  of  the  columns  in  the  dictionary. 

For  example,  the  n  x  n  dictionary  T  is  built  of  the  columns  of  the  n  x  n  DFT  matrix.  The  0 
vector  provides  the  complex  amplitude  values  for  each  vector  used  in  the  signal  x. 

In  Section  2.2,  it  was  shown  that  the  output  of  the  NoLAff  function  is  approximately  linearly 
related  to  the  input  when  the  condition 

IIpII  » llxll 

is  met,  where  p  is  the  {)robe  and  x  is  the  input.  This  led  to  the  expression, 

00 

f(x)  X.  *  y  fl^.p 
*.=1 

This  e<iuation  can  be  developed  into  a  left-hand  factored  form, 

f(x)  «  NlT^,  (9) 

where  Nn  =  •  diag(p)^^'~*K  Here,  Nn  serves  a  similar  role  to  that  of  randomizing  matrices 

used  in  CS-spreading  the  sparse  signal  x.  We  note  that  here  there  is  no  low-pass  filtering  and  hence 
we  recjuire  hypothesis  testing.  CS  techniques  typically  perform  filtering. 


20 


The  NoLAff  function  may  also  be  described  by  a  right-side  factorization, 
y  «  TNRf>, 


(10) 


where  T  =  TNr  is  the  probe-dependent  new  dictionary  resulting  from  applying  the  NoLAff  trans¬ 
formation  to  the  colunms  of  T.  Because  the  output  vector  y  is  assumed  to  be  sampled,  then  the 
aliased  images  all  remain  within  the  space  spanned  by  the  original  dictionary  T.  Thus,  it  is  possible 
to  describe  every  column  of  T  by  a  linear  combination  of  the  columns  of  T.  Here,  Nr  represents 
(approximately)  tlie  spreading  of  itiforraation  caused  by  NoLAff,  and  is  thus  called  the  nonlinear 
spreading  matrix. 

Either  the  left-hand  or  right-hand  factorization  can  replace  the  random  sampling  matrix  in 
the  LASSO  function  for  compressed  sensing,  giving  J{d)  =  j|y  -  ^NlT6/||2  +  A||0|li  and  J(6)  = 
||y-*TNRB||^  +  A||B||,  |8|. 

2.5  SIMULATION  OF  A  NOLAFF  IMPLEMENTATION  IN  AN  RF  SYSTEM 

The  goal  of  this  sectioit  is  to  provide  an  initial  approach  and  analysis  for  integrating  NoLAff 
undersamj)ling  into  an  RF  receiver  system.  -A  basic  system  diagram  for  a  typical  analog  RF  receivt'r, 
shown  in  Figure  12(a),  can  be  converted  to  the  NoLAff  receiver  system  depicted  in  Figure  12(b) 
with  the  addition  of  an  analog  NoL.Aff  subsystem.  The  analog  NoLAff  subsystem  consists  of  a  tone 
generator,  a  summing  block  for  adding  the  strong  probe  tone  to  the  received  signal,  a  nonlinear 
filter  subsystem,  and  several  notch  filters. 

2.5.1  System  components  for  NoLAff  Undersampling 

The  additional  components  found  in  Figure  12(b)  are  required  for  NoLAff  undersampling.  It 
is  the  goal  of  this  section  to  justify  the  inclusion  of  the  extra  components,  and  to  analyze  their 
impact  on  overall  system  [)erformance. 


Probe.  Generator  and  Summing  Block  The  probe  generator  is  required  as  an  essential 
component  in  oi)cration  of  a  NoL.Aff  system.  To  work  w’ell,  the  probe  signal  must  be  much  stronger 
than  the  input  signal  to  whidi  it  is  added.  .A  probe-to-target  ratio  of  60  dB  was  used  for  this 
simulation.  Noise  and  distortions  from  the  probe  generator  will  have  a  comparatively  large  iin])act 
on  system  performance,  and  thus  a  low-noise  and  low-distortion  component  should  be  used.  For  this 
simulation,  a  high-quality  source  was  assumed  and  noise  and  distortion  from  the  probe  generator 
were  considered  negligible. 

The  probe  signal  is  adtled  to  the  target  signal  by  a  summing  block.  The  summing  block  can 
be  iniidemented  with  an  amplifier  w'ith  gain  and  noise  figure  values  similar  to  the  other  amplifier 
stages. 
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Figure  12:  (a)  Basic  block  diagram  for  a  traditional  receiver  system,  (b)  System  diagram  for  the 
hardware  implementation  simulation  for  NoLAff  under  sampling. 

Nonlinear  Filter  Subsystem  The  most  significant  change  to  the  receiver  system  is  the 
inclusion  of  a  nonlinear  filter  subsystem.  This  component  is  critical  to  the  proper  operation  of 
NoLAff  nndersainpling,  and  is  composed  of  four  individual  parts:  a  cubic  rircuit,  an  alTpass  filter, 
a  gain  stage  and  a  summing  block. 


Cubic  Circuit  The  first  component  in  the  nonlinear  filter  subsystem  is  the  cubic  circuit. 
The  function  of  this  circuit  is  to  perform  a  cube  operation  on  the  signal  entering  it.  The  cubic  circuit 
operates  in  the  linear  region  of  all  of  its  components.  The  schematic  for  this  circuit  is  provided  in 
Figure  14.  The  circuit  begins  by  coin^erting  the  signal  voltage  into  current,  then  performs  the 
cubing  o]>eration  in  current.  For  the  last  step,  the  circuit  converts  the  cubed  current  into  a  voltage 
by  passing  it  through  a  resistor. 

SPICE  simulations  were  performed  on  the  circuit  of  Figure  14  to  verify  the  cubic  nature  of 
its  operation.  The  output  frequency  response  to  a  a  single  input  tone  at  25  MHz  and  at  -16.48  dB 
input  power  is  shown  in  Figure  13,  This  shows  the  cubic  circuit  produces  the  expected  3rd-order 
and  fundamental  tones,  and  shows  that  a  weak  5th-order  term  is  also  present.  All  other  harmonic 
tones  were  considered  negligible  for  this  simulation. 

A  further  insight  into  the  operation  of  the  cubic  circuit  is  the  magnitude  and  phase  responses 
from  the  SPICE  simulation  and  included  in  Figure  15.  The  response  curves  show  consistent  behavior 
to  about  200  MHz,  beyond  which  performance  liegrades.  However,  witli  furtlier  design  improvements 


22 


^0(2JX10^ 


r  ^  I  ^ 

^2.71dB) 


■ao^ 


•TBJ^ 


-1CW“ 


I  : 

-123^^1  JJ 


S0.0  ioa,o  10OJO 

Fmr(MHX)  (E6) 


Figure  13:  Output  frequency  response  of  the  cu^ir  circuit  for  an  input  tone  of  25  MHz  and  relative 
input  power  or  -I6.4S  dB. 


Figure  I4:  Diagram  for  the  cubic  circuit 
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Figure  15:  Magnitude  and  phase  responses  of  the  cubic  circuit  with  respect  to  input  tone  frequency. 

The  circuit  performance  begins  to  alter  substantially  above  ^200  MHz. 

to  the  voltage-to-current  and  current-tO' volt  age  converters  at  the  input  and  output  to  the  circuit, 
the  bandwidth  of  the  cubic  circuit  is  expected  to  extend  well  beyond  500  MHz*  Simulation  results 
tleinonstrated  the  cut>ic  circuit  would  he  expected  to  have  a  noise  figure  of  about  6  dB,  (‘Oiisistent 
with  similar  devices* 


All-Pass  Filter  After  the  cubic  c:ircuit,  the  signal  is  filtered  by  an  all-pass  filter.  This 
circuit  provides  a  linear  phase  delay  to  the  signal,  which  frequency-encodes  the  iiitermodnlatioii 
products*  A  passive,  2nd-order  all-pass  circuit  was  used  due  to  its  simplicity  of  implementation  and 
the  relatively  small  amount  of  noise  it  adds  to  the  signal*  Figure  16  provides  a  circuit  diagram  for 
the  passive  second-order  all-pass  circuit.  The  transfer  function  for  this  filter  is  given  by  |9| 


2s{ujq/Q) 

+  s{wo/4?)  +t^o' 


(11} 


The  initial  filter  design  for  NoLAff  calls  for  a  filter  with  a  constant  gain  and  a  linear  phase 
response.  The  2nd-order  all-pass  provides  a  constant  gain  of  0*5  across  all  frequencies  ideally,  and 
up  to  1  GHz  by  simulation*  However,  the  phase  response  is  not  linear  with  frecjuency*  By  adjusting 
the  value  of  Q  in  the  transfer  function  design,  the  phase  response  was  adjusted  to  provide  a  large 
bandwidth  region  with  approximately  linear  operation.  A  SPICE  simulation  of  this  phase  response, 
extending  from  50  MHz  to  250  MHz,  is  shown  in  Figure  17* 
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Figure  16:  Circuit  diagram  for  a  pa.%mje  2nd- order  all-pass  filter  l9f 
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Figure  17:  Spice  simulatio7i  for  the  phase  7'esponse  of  the  2nd- order  all-pass  circuit.  The  I'esponse 
is  approximately  linear  with  fiaquency  in  the  region  from  50  MHz  to  260  MHz. 
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Because  of  the  high  output  impedance  of  the  passive  filter,  a  gain  stage  is  included  at  the 
output.  The  amplifier  also  allows  the  strength  of  the  nonlinearity  to  be  set  to  an  appropriate  level 
before  it  is  added  with  the  linear  pass  through.  This  amplifier  is  followed  by  another  siiiiiiriing 
block,  which  adds  the  noiiliiiearity  signal  to  the  linear  passthrough  signal. 

Notch  Filters  After  adding  the  linear  and  nonlinear  signals  together,  the  last  step  for  tJie 
analog  system  is  to  perform  an  A/D  conversion.  However,  soitie  filtering  should  be  performed  first. 
The  probe  tone  that  was  added  to  the  signal  was  much  stronger  than  the  original  in])ut  signal,  and 
by  itself  does  not  add  any  desired  information  to  the  signal.  So,  in  order  to  improve  the  signal 
dynamic  range  prior  to  the  ADC,  the  probe  tone  is  removed  with  a  notch  filter.  The  harmonics  of 
the  probe  are  also  quite  strong,  and  so  the  3rd  and  5th  probe  harmonic  are  also  notched.  For  this 
simulation,  the  notch  filters  were  implemented  as  ideal,  setting  the  power  at  the  notch  frequency 
to  zero  while  not  affecting  any  other  frequency  locations.  This  recjuired  careful  decisions  as  to  the 
input  signal  frequency  locations,  as  the  signal  and  the  intermodulation  products  should  not  have 
crossed  over  the  notch- filter  frequencies. 


Analog- to* Digital  Converter  The  last  part  of  the  analog  receiver  NoLAfF system  performs 
ail  automatic  gain  function  followed  by  the  analog- to-digital  conversion.  Two  ADCs  were  compared 
for  this  study:  an  ADC  sampling  at  the  full  signal  rate  and  an  ADC  sampling  at  tlie  undersample 
rate.  For  the  full  rate,  a  MAX  108  operating  at  1  Gsps  was  simulated.  The  MAX  108  is  an  8- bit 
ADC,  and  according  to  Maxim,  it  operates  with  7.5  ENoBs  at  1  Gsps.  For  tiie  undersample  rate 
ADC,  the  AnaJog  Devices  AD9461,  which  is  a.  16-bit  converter  with  12  ENoBs  when  running  at  tlie 
sample  rate  of  125  Msps,  was  simulated. 


2.5,2  Derivation  of  System  Noise  and  the  Impact  on  SNR 


One  impact  of  the  additional  components  in  the  NoLAff  receiver  is  the  effect  on  the  noise.  It 
will  be  shown  that  the  system  nonlinearity  will  have  negligible  effect  on  the  overall  system  noise  and 
that  the  low-noise  amplifier  (LNA)  will  remain  the  dominant  noise  source.  Using  Friis’  formula  [10], 
the  noise  figure  up  to  the  second  summing  block,  with  the  nonlinear  branch  removed,  is 


F  F  I  I  I 

Oi  LrlLr2  CjlCr2Cx3LT^ 

while  the  noise  figure  through  the  nonlinearity  with  the  linear  passthrough  removed,  is 


(12) 


Cw  =  Cl  + 
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This  leads  to  a  noise  figure  entering  the  second  summing  block  of 


Csmn2  =  2  (Cl  +  Fn). 


(13) 


(14) 
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Component 

Gain 

Typical  Gain  (dB) 

Noise  Figure 

Typical  Noise  Figure  (dB) 

LNAi 

G, 

12 

3 

Ampi 

G2 

12 

F2 

6 

Mixeri 

Ga 

2 

F3 

8 

Sumi 

G., 

2 

F4 

6 

Amp2 

G5 

30 

Fs 

8 

Cubic 

Ge 

-16 

fe 

-10 

h  filter 

G7 

-6 

Ft 

-6 

Amp3 

Gg 

12 

Fg 

6 

Sum2 

Gg 

2 

Fg 

6 

Amp4 

Gio 

20 

f'm 

8 

TABLE  1 

Gain  and  Noise  Figure  values  for  each  component  in  the  simulated  receiver  sys¬ 
tem  of  Figure  12. 


Siii)st.ituting  gain  and  noise  figure  values  from  Table  1  into  equations  {T2- 14)  gives  Fl  =  3.49  tlB  and 
=  3. 49(iB,  so  Fsnnt2  =  3.49dB  as  well.  Because  the  noise  figure  through  the  linear  pa.ssthrough 
gives  the  same  restilt  as  the  noise  figure  through  the  nonlinearity,  the  circuitry  for  the  nonlinearity 
<'ontributes  no  .substantial  noise  to  the  overall  system.  The  reason  for  this  is  the  higli  gain  provided 
by  -AniiJ'i.  Removing  .4mp>  from  the  system,  for  example,  would  cause  Ffj  to  increase  to  4.84  dB. 
However,  with  it  in  place,  the  LNA,  with  its  3  dB  noise  figure,  is  the  dominant  component  in  the 
receiver  noise. 


2.6  SIMULATIONS  AND  RESULTS 

Three  categories  of  simulations  were  conducted  for  NoL.Alf  hypothesis  testing.  The  first  tested 
the  algorithm  performance,  in  regards  to  ambiguity  error  rate  and  reconstruction  error,  over  a  set  of 
input  signals  that  varied  by  SNR  and  signal  bandwidth.  Tests  were  also  performed  varying  system 
jjarameters,  .such  as  the  .search  bandwi<lth  when  the  actual  bandwidth  is  unknow'ii.  The  secon<l 
.simulations  focu.sed  on  the  performance  due  to  limitations  arising  from  the  use  of  real  system 
components.  An  analysis  of  the  effect  of  different  sources  of  noise  on  hypothesis  testing  is  also 
consklered.  The  final  tests  examined  performance  of  recovering  a  small  signal  in  the  pre.sence 
of  a  largtf  signal.  This  large  dynamic  range  case  presents  sjjecific  challenges  wdiich  other  .A-to-l 
techniques,  specifically  compressed  sensing,  do  not  handle  w'ell.  We  present  promising  results  for 
the  use  of  NoLAff  in  .such  situations. 
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Figure  18:  Ambiguity  error  rate,  verstis  hypothesis  search  bandwidth.  Results  for  a  tone  signal  aiid 
1%,  3%  and  5%  .spectral  occupancy  binary  phdse-.shift-  keyed  (BPSK)  signal  are  shown.  The  lowest 
error  rates  appear  when  the  .search  bandwidth  is  near  the  .signal  bandwidth. 

For  all  of  the  simulations  that  follow,  several  system  parameters  reniaiiiod  consistent  through¬ 
out.  Fir.st,  datasets  began  with  1024  samples  at  full  rate;  an  iindersample  rate  of  8  was  used  for 
all  simulations,  giving  128  samples  in  the  low-rate  data.  The  NoLAff  nonlinearity  was  taken  to  lie 
cubic — or  near-cubic  for  the  hardw'are  simulation — added  to  a  linear  passthrough.  Response  curves 
of  the  filters  used  are  found  in  Figure  6  for  the  algorithm  tests  and  in  Figure  17  for  the  hardware 
simulation.  Other  parameters  and  conditiotis  are  discussed  separately  for  each  simulation  l>elow. 


Search  Bandwidth  Performance  The  first  set  of  results  presentc^d  show  the  performance 
of  NoLAff  hypothesis  testing  for  a  variety  of  input  signals.  Input  signals  were  varied  with  input 
SNR,  ranging  from  5  to  25  dB,  and  w'ith  bandwidth,  ranging  from  a  simple  tone  to  a  BPSK  signal 
occupying  9%  of  the  full  bandwidth.  The  decision  error  rates  over  100,000  trials  for  these  signals 
are  shown  in  Figure  20  as  a  function  of  SNR.  The  reconstruction  MSE  for  this  trial  set  is  |)rovided 
in  Figure  21.  This  shows  the  average  reconstruction  error  for  only  the  correctly  chosen  hypotlieses 
from  the  trial  set.  NoLAff  undersantpling  is  able  to  recover  signals  that  occujjy  a  large  percentage 
of  the  undersample  bandwidth,  as  shown  in  Figures  20  and  21.  At  the  largest  bandwidth  in  tlie 
experiment,  9%,  or  72  %  of  the  folded  spectrum,  NoLAff  achieved  no  less  than  99.9%  accuracy  with 
an  input  SNR  of  at  least  20  dB.  Perforinance  improved  as  the  signal  bandwidth  was  decreased, 
both  in  errant  decision  and  in  reconstruction  error.  The  reconstruction  error  decreased  as  the 
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Figure  19:  RiTMUstmction  en^or  (MSE)  is  shown  as  the  hypothesis  search  bandwidth  increases. 
The  error  is  iowest  for  each  dataset  when  the  search  bandwidth  is  near  the  signal  bandwidth. 


29 


signal  bandwidth  decreased,  which  is  possibly  a  result  of  less  noise  falling  within  the  reconstruction 
bandwidth  used  by  NoLAff. 


Signal  Bandwidth  Performance  Simulation  results  pertaining  to  the  portion  of  the  un- 
dersainpled  spectrum  used  for  hypothesis  testing  and  reconstruction  were  also  collected.  If  the  exact 
bandwidth  of  the  undersampled,  sparse  signal  is  unknown,  then  NoLAff  hypothesis  testing  can  be 
applied  using  a  larger  or  smaller  portion  of  the  measured  bandwidth.  This  is  called  the  search 
bandwidth,  and  the  effects  of  choosing  a  bandwidth  that  was  either  too  large  or  too  small,  on  over¬ 
all  performance  of  NoLAff  undersampling,  were  collected.  Figure  18  shows  the  ambiguity  decision 
error  rate,  and  Figure  19  gives  the  reconstruction  error  for  these  simulations.  Both  figures  clearly 
suggest  that  NoLAff  works  best  when  the  search  bandwidth  covers  exactly  the  hand  which  contains 
the  signal,  as  the  error  rate  and  reconstruction  error  reach  a  minimum  when  the  search  bandwidth 
equals  the  signal  bandwidth.  The  reconstruction  error  shows  a  consistent  pattern  that  repeats  for 
all  bandwidth  trials.  The  pattern  shows  large  errors  when  the  search  bandwidth  is  smaller  than  the 
signal  and  is  likely  due  to  a  portion  of  the  signal  information  being  excluded  from  the  search.  As 
the  search  bandwidth  grows  beyond  the  size  of  the  signal,  the  reconstruction  error  increases  along 
a  very  similar  path  for  all  four  trials.  This  may  be  explained  by  the  inclusion  of  noise  outside  the 
signal  bandwidth  in  the  reconstructed  signal. 


Hardware  Simulation  Performance  The  hardware  simulation  of  Figure  12  was  simulated 
and  compared  to  the  initial  NoLAff  design  and  to  the  full-rate  sampling  of  the  signal.  Results  for  the 
decision  error  rates  and  reconstruction  errors  are  provided  in  Figures  22  and  23,  respectively.  Figure 
22  shows  the  rate  of  errors  encountered  over  a  test  of  100,000  trials  for  several  SNR  values.  Figure  23 
provides  average  mean-square-error  reconstruction  error  over  the  100,000  trials,  but  choosing  only 
trials  where  the  chosen  hypothesis  was  correct.  The  ’initial’  system  has  an  ideal  cubic  polynomial 
filter  with 


'■=[. 
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The  hardware  system  is  based  on  the  hardware  circuit  simulations  for  the 
cubic  nonlinearity  and  the  all-pass  filter.  Two  different  input  signals  were  also  used:  simple  tone 
inputs  and  a  BPSK  signal,  which  occupied  no  more  than  2%  of  the  full  spectrum.  For  all  trials, 
the  probe  signal  was  a  tone  located  at  a  consistent  frequency  location,  and  the  center  frecpiency  of 
the  input  signal  was  chosen  at  random.  Also,  care  was  taken  to  avoid  situations  of  signal  folding  or 
overlapping,  and  the  input  signal  was  chosen  in  order  to  avoid  the  notch  frecpiencies.  Data  for  the 
reconstruction  of  a  full-rate  sampled  signal  was  collected  and  is  provided  in  Figure  23. 


These  results  show  a  trend  similar  to  the  results  found  for  the  algorithm  tests,  with  error  rate 
decreasing  rapidly  with  increasing  SNR,  and  average  reconstruction  error  improving  with  increasing 
SNR.  However,  a  surjmsing  result  was  found  when  comparing  the  hardware  error  rate  to  the  initial 
design  error  rate  in  Figure  22:  the  hardware  simulation  outperformed  the  initial  system  design, 
showing  a  lower  error  rate  at  all  SNR  values.  This  indicates  several  points.  First,  the  differences 
between  the  hardware  and  initial  design  performace  are  most  likely  related  to  the  encoding  scheme. 
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Fitjur^e  20:  NoLAff  undersampling  error  mte  over  100^000  trials  for  tone  signals  and  i%,  0%^  5%, 
7%f  and  9%  spectml  occupancy  BFSK  signals. 
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NoLAff  Undersampling  Performance:  MSE  (Input  vs.  Correct  Hypothesis) 


Figure  21:  Reconstruction  error  (MSE)  for  NoLAff  undersarnpling  of  tone  signals  and  1%,  3%, 
5%,  7%  and  9%  spectral  occupancy  BPSK  signals^ 


Figui^  22:  Ambiguity  error  jutes  for  hardwaj^e  and  initial  design  simulations.  Error  rates  are 
shown  for  both  tone  signals  and  2%  spectral  occupancy  BPSK  signals. 
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Figure  23:  Mean  square  error  for  signal  reconstruction  in  hardware,  initial  design  and  full-mte 
siniulatiojis.  MSE  for  both  tones  and  a  2%  spectral  occupancy  DPSK  signal  is  shown. 
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which  is  controlled  by  the  all-pass  filter  within  the  nonlinearity.  Second,  this  suggests  that  neither 
the  initial  design,  nor  the  hardware  simulation,  implements  an  “ideal”  encoding  scheme.  The  ex¬ 
planation  of  NoLAff’  encoding  provided  in  Figure  7,  while  instructive,  is  overly  simplistic  and  does 
not  accurately  reflect  the  more  complicated  interactions  between  the  filtered  nonlinear  distortions, 
the  linear  passthrough  signals,  and  the  effects  of  iterative  NLEQ.  Exploring  the  impact  of  different 
group  delays,  nonlinear  phase  delays  and  phase  offsets  is  important  future  work,  whi(;h  should  lead 
to  performance  gains  by  reducing  overall  error  rates  for  both  software  and  hardware  system  ilesigns. 


Noise:  LNA  vs.  ADC  Two  components  in  the  receiver  system  are  expected  to  dominate 
the  noise-the  Low  Noise  Amplifier  (LNA)  and  the  Analog-to-Digital  Converter  (.4DC).  The  ADC 
is  expected  to  dominate  when  the  signals  present  in  the  measured  data  have  a  large  dynamic  range 
requirement;  the  LNA  is  expected  to  dominate  otherwise.  .Any  steps  that  can  be  taken  to  low'cr 
the  dynamic  range  of  the  signals  help  reduce  the  effect  of  the  -ADC,  and  this  is  why  the  probe  and 
its  harrnonics-the  strongest  signals  present  in  the  system  were  removed  with  analog  notch  filters 
prior  to  sampling.  When  the  .ADC  is  the  dominant  source  of  noise,  NoL.Aff  can  produce  stunning 
improvement  in  overall  SNR  by  allow'ing  a  hwer-rate  converter  to  be  used,  gaining  as  much  as  30  dB 
in  dynamic  range.  Figure  24  demonstrates  that  the  low’er-rate  .ADC  reduces  the  noise  caused  by  the 
converter,  leading  to  improvements  in  SNR.  The  benefits  of  NoL.Aff  when  the  LN.A  is  the  dominant 
component  are  not  as  dramatic,  but  do  show'  some  improvement  as  shown  in  Figure  23.  How'cver, 
w'ith  the  gains  in  SNR  offered  by  the  NoL.Aff  approach,  .some  of  the  regained  SNR  is  retpiired  in 
order  to  make  correct  decisions  for  the  signal  reconstruction. 


Large  Dynamic  Range  Environments  Detection  and  reconstruction  of  a  weak  signal 
from  a  spectrum  crow'ded  with  stronger  signals  is  a  <liffictilt  problem  arising  in  many  real-world 
applications.  NoLAff  undersampling  is  explored  as  a  method  to  facilitate  the  removal  of  a  strong 
interfering  signal,  allowing  greater  access  to  a  w'eaker  signal.  The  improved  dynamic  range  from 
undersampling  the  signal,  depicted  in  Figure  24,  provides  improvement  in  SNR.  In  CS  approat'hes 
ba.sed  on  random  projections,  the  re.sidual  error  of  reconstructerl  large  signals  is  effectively  noise  for 
small  signal  detection  and  reconstruction.  NoL.Aff  applied  properly  does  not  have  this  proiilem. 

Let 


X  =  Xs  -l-Xw, 


be  the  received  signal,  wdiere  Xg  and  Xw  are  sparse  signals,  non-overlapping  in  fretiuency,  and 
Ijxsil  »  ||xwl|.  The  input  signal  is  encoded  by  the  NoLAff  system  as  described  in  Section  2.2.  The 
decoding  strategy  is  to  first  remove  the  strong  signal  Xg,  then  redefine  the  probe  as  Pp^  s  =  P  +  Xg 
to  resolve  the  w'eak  signal  Xw-  Re.sults  are  provided  in  Figures  25  and  26.  Figure  25  show's  the 
ambiguity  error  rate  for  choosing  the  correct  hypothesis  for  the  w'eak  signal  for  a  given  SNR  for  the 
weak  signal.  The  strength  of  the  strong  signal  varies  from  10  to  50  dB  above  the  w'eak  signal,  and 
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Large  DR  -  ADC  Dominates 


Figure  24:  The  irlationship  of  ADC  noise  and  LNA  noise  to  overall  system  SNR.  The  top  di¬ 
agram  shows  that  for  signals  with  large  dynamic  range,  the  lower-rate  ADC  afforded  by  NoLAff 
undersampling  7'esults  in  lower  ADC  noise  and  an  overall  improvement  in  SNR.  For  signals  with 
a  smaller  dynamic  range,  as  in  the  lower  diagram,  an  Automatic  Gain  Controller  is  a.s.sumed  to 
boost  the  .signals  and  LNA  noise  prior  to  the  A  D  conversion,  and  improvements  in  ADC  noise 
do  not  have  a  substantial  effect  on  the  overall  SNR. 
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Figure  25:  Decision  error  rate  for  a  weak  signal  with  a  sti^ong  signal  present  is  plotted  for  the 
SNR  of  the  weak  signal,  and  for  the  dynamic  mnge  between  the  strong  and  weak  signals.  Also 
provided  for  comparison  is  the  error  rate  when  there  is  no  large  signal  piosenL  Enor  rates  closely 
match  the  case  with  no  strong  signaL  Large  dynamic  range  cases  begin  to  exhibit  inaccuracies 
from  insufficient  iterations  of  the  iterative  NLEQ. 


the  probe  was  set  to  90  dB  stronger  than  the  weak  signal.  Performance  for  the  w^eak  signal  recovery 
in  the  presence  of  the  strong  signal  closely  matches  that  of  the  weak  signal  alone.  At  the  larger 
dynamic  range,  the  effect  of  the  iterative  NLEQ  inaccuracy  can  be  seen  by  an  increase  in  the  error 
rate.  Increasing  the  number  of  iterations  would  likely  improve  performance  at  the  higher  dynamic 
range. 

Figure  26  presents  the  large  dynamic  range  reconstruction  error  results.  These  results  include 
only  the  cases  where  the  correct  hypothesis  signal  is  successfully  cliosen.  Reconstruction  of  a  weak 
signal  is  not  significantly  affected  by  the  presence  of  a  strong  signal  once  the  correct  hypothesis  is 
chosen. 
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FiguiT  26:  The  rttconsb'^tction  error,  as  avemge  mean  square  e^ror,  is  provided  for  weak  signals 
i7i  the  pt'e.sence  of  a  stronger  signal.  This  suggests  the  strong  signal  has  little  to  no  effect  on  the 
reconstruction  of  a  weak  signal  once  it  has  becM  eoTrecUy  identified  from  the  hypothesis  signals. 
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3.  VARIABLE  PROJECTION  AND  UNFOLDING 


Basis  Pursuit  (BP)  is  remarkably  efficient  in  reconstructing  signals  that  randomly  occupy  a 
small  fraction  of  the  positions  in  the  discrete  basis  where  the  signals  have  a  sparse  representation. 
However,  we  have  fo\md  that  in  dense  signaling  environments,  i.e.,  environments  in  which  the  signals 
occupy  more  than  15-20  percent  of  the  downsampled  basis,  BP  is  unable  to  effectively  reconstruct 
the  original  signal.  To  reconstruct  not-so-sparse  signals  that  occupy  up  to  nearly  100  percent  of 
tlie  dowuisampled  basis,  we  have  developed  Variable  Projection  and  Unfolding  (VPU),  which  is 
a  maximum  likelihood  (ML)  sequential  detection  and  estimation  technique  that  searches  for  and 
identifies  the  contiguous  columns  in  the  discrete  basis  matrix  that  the  signals  span.  VPU  is  similar  to 
orthogonal  matching  pursuit  (OMP)  |11|;  however,  unlike  OMP  that  searches  over  rank-1  subspaces 
to  identify  the  ba.sis  suj>port,  VPU  is  capable  of  searching  over  subspaces  of  any  rank.  In  this  section, 
we  {jresent  the  \TU  algorithm  and  its  computational  complexity  and  highlight  the  results  of  sen.sor 
network  simulations  comparing  VPU  and  BP  peformance.  In  Section  5,  we  derive  and  compare  the 
ML  niinimnm-mean-sqtiare-error  (MMSE)  bounds  to  BP  bounds  to  support  the  results  presented 
here. 


3.1  VARIABLE  PROJECTION  AND  UNFOLDING 

Variable  Projection  and  Unfolding  is  similar  to  OMP  as  both  first  try  to  directly  iilentify 
the  basis  support  of  the  received  signal.  OMP  proceeds  by  first  correlating  each  of  the  columns 
of  the  basis  with  the  received  signal  and  selects  the  column  that  has  largest  absolute  correlation. 
Using  the  selected  basis  column,  the  original  signal  (prior  to  randomization  and  downsampling) 
is  reconstructed  and  subtracted  from  the  received  signal,  and  the  basis  column  is  added  to  the 
reconstruction  basis  column  set.  This  process  continues  iteratively  as  illustrated  in  Figure  27. 
-Although  computationally  efficient,  the  principal  drawback  to  OMP  is  its  restrictive  search  over 
rank-1  subspaces  which  preclude  it  from  exploiting  the  correlation  between  coefficients  in  the  basis 
in  which  the  signal  has  a  sparse  representation. 

VPU  is  nearly  identical  to  OMP  with  the  exception  that  the  search  process  is  not  restricted  to 
identify  ba.sis  support  only  over  rank-1  subspaces.  VPU  sequentially  searches  over  rank-ra  subspares 
to  identify  the  basis  sujtport.  .As  opposed  to  correlation,  VPU  projects  the  signal  onto  the  subspace 
si>aimed  liy  the  candidate  n  cohimns.  If  the  magnitude  of  the  projection  (mean  square  error)  is  below 
a  prescribed  threshold,  the  candidate  columns  are  added  to  the  reconstruction  basis  set,  and  this 
set  will  be  used  to  form  new  projections  when  search  commences.  V^PU  proceeds  by  sequentially 
moving  to  a  new  raiik-n  subspace  (e.g,,  adjacent  or  overlapping  basis  cohimns)  and  periodically 
increinentiiig  the  subspace  size.  A'PU  processing  is  graphically  illustrated  in  Figure  28.  Unlike 
OMP,  VPU  exploits  the  correlation  between  coefficients  to  enable  it  to  significantly  outperform 
either  OMP  or  BP  in  reconstructing  signals  that  occupy  more  than  15  percent  of  the  downsampled 
basis,  which  is  demonstrated  in  this  section.  The  details  of  the  algorithm  are  presented  below. 
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Matching  Pursuit 


Compressive  Receiver 
y  s  (l>Tx  +  V 

A 


Sequential  detection  and 
estimation 

“  Sequential  projective 
subtraction 

Computationally  efficient 
Search  for  basis  support  is 
over  rank-1  subspaces 
Less  robust  in  identify^ing 
basis  support  than  BP 
-  e,g.,  weak  sfKctrai 
componer^ts  in  wideband 
signal  not  highly  correlated 
with  received  signal 


Figure  27:  Block  diagimn  of  orthogonal  matching  pursuit  processing. 


Variable  Projection  and  Unfolding 


Compressive  Receiver 

y  =  04'X  4*  V 

A 

•  Sequential  detection  and 
estimation 

-  Sequential  projective 
without  subtraction 

•  Computationally  less 
efficient 

•  Search  for  basis  support  is 
over  subspaces  of  any  rank 

•  More  robust  in  identifying 
basis  support  than  BP 

-  e,g.,  weak  spectral 
cornponente  in  wideband 
signal  highly  correlated 
With  receive  signal 


Figuje  2S: 


Block  diagram  of  VFU  processing. 
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Variable  Projection  and  Unfolding  (Coarse  Search) 

s_cols  =  {0},  search  =  0; 
while  (search  <  signals_present) 

{ 

for  (chunk  =  min_swath;  chunk  =  chunk  +  P;  chunk  s  max_swath) 
s  =  {1,2,— .chunk} 

for  (slide  =  0;  slide  s  N;  slide  =  slide  +  chunk) 

{ 

error[slide,chunk]  Emmse  fo'’  union(s+slide,  s_cols): 
[min_slide,  min_chunk  j  argmin  error 

I  slide,  chunk 

s_cols  =  union(s_cols,  (1,2,...,  min_chunk}+min_slide}; 
search  =  search  +  1 ; 

} 


Figure  29:  VPU  pseudocode  for  identifying  and  reconstructing  signals  using  a  coarse  se.airh. 

Consider  a  signal  x  G  occupying  unknown  frequency  locations  in  some  bandwidth  B 

that  is  digitizc'd  at  a  rate  2B  D,  with  D  =  B'P  where  ib  €  is  the  basis  matrix  in  which  x  has 

a  sparse  representation,  d  is  the  downsample  factor  and  BERL^J^^isa  randomizing  downsample 
matrix  \12\.  The  digitized  signal  can  then  be  described  by  y  =  Dx-\‘Dv  where  the  noise  v  6  rI-'^J  ^  ^ 
and  signal  are  normally  distributed  as  v  ~  ^^(0,  Cy)  and  x  ^  M{x,  Cx),  respectively,  and  the  symbol 
represents  the  greatest  integer  that  is  less  than  or  equal  to  z,  VPU  is  capable  of  reconstructing 
not-so-sparse  signals  in  the  presence  of  noise,  where  the  term  not-so-sparse  refers  to  signals  that 
can  occupy  up  to  elements  in  the  basis  in  which  the  signal  is  sparse.  VPU  is  a  sequential 

estimation  and  detection  process  which  searches  over  the  columns  s  =  [^l.s 

the  reconstruction  error  that  minimizes 

arguyn  ||£-a/a/.S£:||2  =  arginin  ||(/  -  D{s))y\\2 
whore  Dis)  =  [D].,  ([D]"  [D],  +  C-^)-*  [D]"  C"*. 

The  pseudocode  for  \TU  is  listed  below.  VPU  sequentially  searches  for  the  columns  that  minimize 
the  error  in  equation  (15);  the  columns  of  D{s)  that  minimize  the  error  in  (15)  are  then  fixed  and 
the  search  continues  for  other  signals  present. 

Note  that  the  coarseness  of  the  search  is  controlled  by  the  three  parameters:  min_swath,  13, 
and  max_swath  in  which  max_swath  is  typically  set  equal  to  [^J  ,  and  the  parameter  /i  controls 
how  many  excess  locations  in  the  signal  basis  are  used  in  the  detection  and  estimation  process. 
A  large  /i  translates  into  reduccxl  sensitivity  due  to  the  inclusion  of  excess  noise,  while  a  small 
imprcnes  detection  and  estimation  performance  at  the  expense  of  computational  complexity.  Once 
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VPU  (Recursive  Fine  Search) 

signal  =  1 ;  s  =  s  -  perturb;  error  =  0; 

double  fine_search (signal,  signals_present,  s,  error) 

{ 

if  (signal  S  signals_present) 

{ 

for  (offset  =  0;  offse1=offset+1 ;  offset  s  2xperturb) 

{ 

error  =  tine_search(signal+1,  signals jaresent,  s,  error) 
s(signal)  =  s(signal)  1 ; 
if  (signal  ==  signals_present)  { 
error{config(s))<-  find  s  that  minimizes  EmmseJ 

} 

} 

1 

} 

s  associated  with  min(error(config(s))); 


Figure  30:  VPU  pseudocode  for  refining  tfte  identification  atid  reconstmction  of  signals  located  in 
coarse  search. 

ail  the  signals  are  found,  a  fine  search  is  conducted  by  jointly  varying  the  column  widths  of  D{s} 
that  the  detected  signals  span  over  a  very  small  search  radius  to  find  the  minimum  error. 

For  cases  where  the  channel  coefficients  and  signal  are  unknown  and  for  relatively  high  signal- 
to-noise  ratio  (SNR),  equation  (15)  can  be  simplified  using  D{s)  «  [Dj^  [J?],,)  ^ 

so  that 


argmhilleA/MSElii  ^  argimn  ;i/^/^-^|^(s)j/ 


(IG) 


where  =  (/  —  t^ls  the  projector  onto  the  null  space  of  the 

columns  of  [D]g  .  Because  the  projector  in  (16)  is  varied  over  the  columns  of  the  matrix  D{s),  the 
signal  is  detected  and  reconstructed  via  variable  projection  and  unfolding  by  locating  the  cohiinn 
positions  in  the  basis  matrix  that  the  signal  vector  spans  prior  to  downsampling  and  aliasing. 

Once  the  column  locations  that  the  signal(s)  span  have  been  identified,  equation  (16)  can  be 
reformulated  for  joint  synchronous  center  frequency  estimation  and  baseband  signal  reconstruction. 
.4s  an  example,  consider  the  simple  case  of  a  single  signal  that  is  sparse  in  the  frequency  basis,  then 
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+  Dr 


[^idft]s 

' - V - 

witii  0  =  tail”'  and  x  =  0.5  • 

\^2  J  'COS0  sin0'’ 


y  = 

e  0  1 

cos 

j 

—  sin  u/’A-n 

[^/DFt]s 


(17) 


where  the  carrier  phase  offset  4>  i  baseband  signal  x  and  center  frequency  are  sinniltaneously 
recovered  using  fine  search  \'PU  over  where  the  initial  estimate  of  uik  is  the  frequency  associated 
with  center  colunin  of  [^/oFr]s  •  Extending  (17)  to  handle  multiple  signals  is  a  straightforward 
extension  of  the  formulation  above. 


TIk'  cotnputational  complexity  of  \'PU  is  dominated  by  iterative  matrix  inversion  requiring 
()(A-  •  q^)  operations  per  iteration,  with  k  =  max_swkth-(y i).min_swath  ^  «ig„ais_p„.sent 

iterations.  The  computational  complexity  of  V’PU  is  higher  than  basis  pursuit  (BP)  0{k'f^p  • 
and  matching  pursuit  (MP)  ()(  -N^q)  |11|,  where  k[^p  is  the  number  of  iterations  in  the  basis 

pursuit  optimization.  However,  we  will  show  that  the  \TU  is  capable  of  reconstructing  signals  that 
occupy  up  to  positions  of  the  discrete  signal  basis  while  both  MP  and  BP  fail  far  before 

reaching  this  limit. 

As  demonstrated  in  the  next  section,  BP  is  very  well  suited  for  identifying  and  reconstructing 
signals  that  occupy  only  a  small  fraction  of  signal  basis,  while  \TU  is  capable  of  identifying  and 
reconstructing  signals  that  occui)y  a  significantly  higher  fraction  of  the  basis.  It  is  interesting  to 
consider  combining  \TU  and  BP;  the  more  computationally  efficient  BP  would  run  initially  and 
identify  sparse  signals  in  the  basis;  \TU  would  build  on  BP’s  coarse  identification  and  extend  the 
capacity  of  .\2I  to  reconstruct  signals  well  beyond  the  BP  limits  as  expressed  in  eejuation  (51). 
Combining  BP  and  \  PU  is  currently  an  active  area  of  research. 


3.2  VPU  PERFORMANCE 

Two  simulation  scenarios  were  used  to  test  the  efficacy  of  \TU  and  to  compare  its  basis  support 
identification  and  reconstruction  performance  against  BP.  In  the  first  test  scenario,  \TU  and  BP 
detected  and  reconstructed  a  random  signal  without  noise  that  occupied  from  8  to  62  contiguous 
positions  in  a  52()-dimensional  frequency  basis,  i.e.,  with  9  6  ^  .  The 

second  simulation  scenario  modeled  a  signal  intelligence  (SIGINT)  Ku-band  receiver  operating  in 
an  environment  with  one  to  three  binary  phase-shift  keyed  (BPSK)  signals  present  and  power  levels 
at  the  receiver  that  ranged  from  -67  dBm  to  -77  dBm  in  a  500  MHz  band,  yielding  roughly  a  10 
dB-2()  clB  SNR.  The  receiver  employed  an  .ADC  with  a  sampling  rate  of  125  MHz;  in  all  cases 
the  aggregate  BPSK  signal  spectral  occupancy  was  52  .MHz  with  each  of  the  BPSK  transmitters 
employing  a  pulse  shaping  root-raised  cosine  filter  with  a  roll-off  factor  of  0.3.  For  all  scenarios  we 
ran  ten  thousand  Monte  Carlo  simulations  and  measured  the  performance  of  both  VPU  and  BP; 


43 


for  BP  we  used  an  ^2  ~  mixed-norm  optimization 

arg  mhi  ||i/  -  +  A  ||a*||i ,  (18) 

where  tlie  regularization  parameter  A  was  chosen  to  optimize  BP  performance  |13|.  In  all  cases  the 
randomizing  downsampling  matrix  0  had  eight  uiiiciue  elements  per  row  that  were  chosen  from  an 
i.i.d.  Gaussian  distribution. 

3.2.1  Test  Scenario  1:  Identification  and  Reconstruction  of  Noiseless  Wideband 
Signal 

The  performance  results  for  the  first  test  scenario  are  tabulated  below. 


8  locations 

16  locations 

24  locations 

62  locations 

BP 

100% 

62% 

17% 

0% 

VPU 

100% 

100% 

100% 

O 

O 

TABLE  2 

BP  and  VPU  identification  performance  with  a  single  noiseless  wideband  signal 
occupying  exactly  8, 16,  24  and  62  frequency  locations  in  a  520-point  frequency 
basis  with  a  downsampling  factor  of  8. 


Identification  performance  in  Table  2  corresponds  to  the  percentage  of  times  in  our  simulations 
that  BP  and  VPU  correctly  identified  all  of  the  locations  in  the  frequency  basis  that  the  signal 
randomly  occupied.  Note  that  in  all  cases  VPU  was  able  to  simultaneously  identify  and  reconstruct 
the  signal  with  virtually  perfect  performance.  .\s  will  be  predicted  in  (51),  BP  never  was  able  to 
perfectly  identify  the  freciuency  support  and  reconstruct  signals  that  occupied  any  more  than  ^ 
of  the  downsampled  Nyquist  band.  .Although  VPU  was  able  to  perfectly  identify  and  reconstruct 
wideband  signals  that  occupied  vip  to  62  positions  in  the  frequency  basis,  it  was  unable  to  uniciuely 
identify  the  frequency  support  when  the  signal  occupied  64  frequency  positions.  This  is  because 
the  projector  in  (16)  becomes  an  all-zero  matrix,  making  it  impossible  to  discriminate  which  basis 
locations  used  in  reconstruction  minimize  the  MSE.  Using  ML  or  MMSE  techniques  for  simultaneous 
identification  and  reconstruction  means  that  the  signals  need  to  occupy  no  more  than  one  (for 
single-sided  spectrum)  or  two  (double-sided  spectrum)  fewer  positions  from  the  entire  downsanijjletl 
Nyquist  band  to  successfully  identify  the  basis  support. 

3,2.2  Test  Scenario  2:  SIGINT 

The  identification  and  reconstruction /demodulation  performance  of  the  SIGINT  receiver  is 
illustrated  in  Figures  31,  32,  and  33  below.  In  Figure  31,  we  illustrate  the  basis  support  misiden- 
tification  rate  of  VPU  for  the  cases  where  one,  two  or  three  BPSK  signals  were  present  and  whose 
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probability  of  misidentifying  frequency  support 


SNRindB 


Fufure  31:  The  mte  of  the  rnisidejitification  of  the  frequency  support  of  BPSK  signals  occupying 
(171  aggregate  of  52  MHz  in  the  frequency  basis.  The  500  MHz  bajidwidth  j^ceived  was  downsafupled 
by  a  factor  of  S. 

aggregate  spectral  occupancy  prior  to  the  3  dB  pulse-shaping  filter  roll-off  was  approximately  52 
MHz.  BP  performance  was  difficult  to  gauge  as  it  was  incapable  of  isolating  only  those  frequencies 
sj)anned  by  the  BPSK  signals  present.  Given  correct  frequency  support  identification,  the  measured 
deinoduiation  hit  error  rate  (BER)  after  reconstruction  was  compared  to  the  theoretical  optinuim 
performance  of  an  ML  soft  “decision  demodulator.  Figure  32  illustrates  the  difference  hetweem  theo¬ 
retical  optimum  and  VPU  performance  with  the  gap  graphically  illustrating  tlie  effects  of  the  folding 
loss,  in  Figure  32,  VPU  BER  performance  is  averaged  over  reconstructing  from  one  to  three  BPSK 
signals,  while  for  BP  the  average  is  only  over  one  reconstructed  BPSK  signal;  this  is  hoeause  after 
one  signal  BP  performance  was  not  noticeably  bettc^r  than  random  selection.  Finally,  Figure  33 
illustrates  BPSK  reconstruction  at  a  digital  carrier  frequency  of  tt/S  and  an  SNR  of  15  dB  jjrior  to 
carrier  frecpieiK'y  estimation  and  synchronous  baseband  demodulation  using  equation  (17). 


3.3  WIDEBAND  CHIRP  SIGNAL  WITH  INTERFERENCE 

Like  BP,  V’PU  is  capable  of  identifying  and  reconstructing  signals  in  bases  other  than  freciuency. 
As  an  example,  consider  the  test  scenario  for  the  identification  of  a  chirp  signal  illustrated  in  Figure 
34.  A  500  MHz  linear  FM  chirp  signal  was  received  in  the  presence  of  two  strong  narrawbantl 
interferers.  The  signal- to-interference4  noise  (SINK)  ratio  was  -1  dB.  The  received  signals  were 
subject  to  random  linear  jjrojection  at  the  RF  front-end  and  digitized  at  a  rate  of  250  MHz,  We 


45 


Figure  S2:  BER  after  BPSK  demodulation  using  VPU  and  BP  compared  to  theoretical  optimum. 
Note  that  the  perfoiinance  gap  between  the  optimum  and  VPU  in  due  to  the  folding  lorn. 


VPU  demodulation  performance;  15dB  SNR 

2  : 
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Figw'e  3S:  Example  of  VPU  reconstf'uction  of  a  BPSK  signal  centered  at  the  canier  frequency 

7:/8. 
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500MHz  linear  chirp 


Normalized  Frequency  (xtc  rad/sample) 


FiguiT  34:  Chirp  signal  at  input  to  the  receiver  prior  to  randomization  and  downsampling  in  the 
presence  of  two  strong  naiTowband  interfere.rs  with  SI  NR  of  -1  dB. 

us(h1  \TU  to  identify  the  loeations  in  the  chirp  basis  that  yielded  the  lowest  MSE.  In  10, 000  Monte 
Carlo  runs,  \TU  always  correctly  detected  and  identified  (set*  Figure  35)  the  chirp  signal  buri(*d  in 
interference  plus  noise  at  the  output  of  the  analog-to-digital  converter. 
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reconstructed  amplitude 


Figure  35:  Detection  of  a  500  MHz  chirp  signal  in  the  presence  of  two  strong  najTowhand  inter- 
ferers. 
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4.  IMPLEMENTATIONS  OF  COMPRESSED  SENSING  USING 

DYNAMICAL  SYSTEMS 


This  section  treats  the  use  of  dynamical  systems  for  compressed  sensing  in  sparse  signal 
environments.  In  applying  the  algorithmic  techniques  of  compressed  sensing  to  RF  sensors,  we  utilize 
analog  signal  processing  that  incorporates  randomization  and  memory.  The  processed  analog  data 
are  sampled  at  a  rate  that  is  typically  well  below  Nyquist,  but  full-band  reconstruction  of  signals  is 
possible  given  sparse  signal  environments. 

Dynamical  systems  provide  a  natural  model  for  analog  signal  processing  w'ith  the  re<juisite 
memory  and  randomne.ss.  A  typical  dynamical  system  is  characterized  by  an  ordinary  differential 
etjuation  (ODE)  that  models  system  behavior.  Sampled  outputs  of  the  dynamical  system  can 
provide  initial  conditions  for  the  ODE  that  allow  the  estimation  of  intersample  values,  supporting 
the  concept  of  sampling  below  Nyquist  rates. 

When  the  <lynamical  system  is  driven  by  an  unknowui  input  signal,  sampled  values  can  be 
used  to  form  an  innovation  process  that  is  modeled  by  the  ODE  of  the  dynamical  system.  Given 
sparseness  of  the  input  in  some  known  basis,  we  can  use  the  sampled  values  to  reconstruct  the  input 
signal.  The  precise  manner  in  which  we  can  achieve  this  reconstruction  is  discussed  below. 

There  is  an  extensive  literature  on  dynamical  systems  outside  of  the  present  context.  A  few 
book  tK'atments  are  mentioned  here.  A  standard,  fairly  complete,  modern  reference  is  (14|.  The 
nonlinear  nature  of  dynamical  systems  suggests  that  the  study  of  specific  cases  can  provide  in.sight 
into  more  general  behavior.  Both  |15|  and  |16|  provide  numerous  useful  examples.  A  theory  of 
the  global  behavior  of  dynamical  systems  has  been  a  work  in  progress  for  a  number  of  decades.  .A 
collection  of  significant  results,  emphasizing  entropic  properties,  is  presented  in  [17]. 

Section  4.1  discusses  discrete-time  dynamical  systems,  illustrating  the  randomization  aiul 
memory  that  are  important  for  compressed  sensing.  The  role  of  dynamical  systems  in  compressed 
sensing  is  described  and  lays  the  foundation  for  the  practical  physical  realizations  described  later 
in  Section  4.2.  Section  4.3  describes  the  compressed  sensing  algorithms  used  in  the  examples.  Se<‘- 
tion  4.4  presents  a  practical  realization  of  a  low-power,  wideband  dynamical  system.  The  technhjues 
introduced  in  Section  4.2  are  used  as  tools  in  modeling  this  system,  which  is  described  by  a  delay 
differential  equation. 


4.1  RELATIONSHIP  OP  DYNAMICAL  SYSTEMS  TO  COMPRESSED  SENSING; 
DISCRETE-TIME  SYSTEMS 

Discrete-time  dytiamical  systems  are  based  on  the  deterministic  sequence  j„+i  =  f{x„)  gen¬ 
erated  by  the  nonlinearity  fix).  For  many  choices  of  /,  including  those  motivated  by  physical 
realizations,  the  setpience  {j;„}  ha.s  an  apparent  statistical  behavior  that  can  be  related  to  that  of 
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a  limiting  form  of  the  random  process  =  f{Xr,{y))  +  where  {y„  }  is  stochastic.  When 

{y„}  approaches  0,  the  limiting  statistical  behavior  can  be  attrilmted  to  the  deterministic  {xn(0)}. 
The  random  process  {;c„{3/)}  is  described  physically  as  a  driven  form  of  the  deterministic  dynamical 
system  with  drive  {j/n}-  hi  the  continuous  case,  such  random  processes  can  be  associated  with 
random,  Markovian  flow's  on  a  manifold  and  have  an  Ito-like  calculus. 

Detection  and  estimation  of  {y„}  can  be  perfonned  in  a  differential  sense  based  on  the  obser¬ 
vations  {x„(2/)}.  If  the  dynamical  system  determined  by  f{x)  is  highly  structured  (e.g.,  has  zero 
entropy),  one  can  work  instead  with  a  partially  knowm  drive  of  the  form  )/„  —  y„  +  tl’n^  wdiere  y„  is 
known. 

Some  benefits  of  differential  detection  and  estimation  using  a  dynamical  system  involve  the 
ability  to  work  with  subsampled  data,  given  certain  assumptions  on  the  input  {;(/„  }■  For  conve¬ 
nience,  we  avoid  a  statistical  model  for  {j/n  }  here  and  treat  the  observations  deterministically  ami 
perturbatively. 

To  see  the  benefits  of  the  dynamical  system  and  the  connection  wnth  compressetl  sampling, 
consider  subsampling  the  data  by  observing  only  for  fixed  n  and  I.  Iterating  the  mapping 

/(x)  /  times,  we  obtain  the  sequence  of  innovations 


I  times 

p„+„il  =  X.r,+ml{y)  -  /'(x„+(m_i3<(y))  =  X„+mi  -  (/  0  . . ,  0  (;(/})■  (19) 

which  represents  the  cliffereiice  between  the  observed  datum  Xn+mi  <tnd  the  predicted  clatuni  Inised 
on  the  last  observation  Linearizing  about  the  deterniinistic  solution  /(())},  one  has 

the  approxiniation 


;-i 


where 

4  f  n!t=l+l  /'(/'" (^n+(m-l)f))  0  <  .S  <  /  -  2 

S  =  f-1 

The  latter  can  be  summarized  in  the  compact  form 


p  =  x-  f{x)  ^  A-  y, 


(20) 


(21) 


(22) 


given  a  finite  vector  x  *-*  {xn+mii}nf=i  of  sulisampled  observations  and  a  corresponding  finite  input 
vector  y  «-*  Here,  the  matrix  A  represents  the  linearized  relation  betw'een  tlie  input 

{y„)  and  the  innovations  x  <-»  {pn+m(}m=i-  ^  is  a  rectangular  matrix  with  more  columns  than 
rows  (A/  X  Ml),  reflecting  the  subsampling.  The  notation  /(x)  indicates  coiuponent-by-component 
application  of  /. 
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Figuiv  36:  The  hifurvation  diagrum  of  the  logistic  system  is  shoxtm.  Each  point  on  the  plot  belongs 
to  a  periodic  orbit.  i4s  the  bifurcation  parameter  increases,  the  oji)it  shifts  location  until,  at  certain 
thresholds,  it  doubles  m  letigth. 

Calculating  A  rcKjuires  only  the  subsampled  observations  expressed  by  x.  However,  these  obser¬ 
vations  depend  on  y  implicitly.  Thus  the  apparently  linear  equation  (22)  is  in  fact  highly  nonlinear. 
However,  equation  (22)  can  be  used  along  with  sparsity  models  for  ff  to  solve  the  underdetennine«l 
system. 

The  dynamical  system  associated  with  the  logistic  map  defined  by 

f{.r)^rx{\-x)  (23) 

luis  been  used  to  represent  population  dynamics  of  biological  systems  (16|  from  epoch  to  epoch. 
The  parameter  r,  0  <  r  <  4,  characterizes  a  family  of  systems  that  exhibit  substantially  different, 
but  stable,  behavior.  The  logistic  map  has  a  single  attracting  fixed  point  at  zero  for  ()<  /•<  1. 
As  r  increases,  an  additional  stable  fixed  point  appears  and,  with  further  increases  in  r,  bifurcates 
into  a  stable,  attracting  orbit  of  two  [)oints.  Further  increases  in  r  lead  to  more  and  more  orbit 
doubling  bifurcations  until,  at  around  r  =  3.6,  the  dynamical  system  exhibits  quasi-ramlorn,  but 
stable  behavdor.  The  bifurcation  diagram  (the  traces  of  points  involved  in  periodic  orbits)  is  shown 
schematically  in  Figure  36. 

Some  simple  examples  of  the  performance  of  a  discrete-time  dynamical  system  are  shown 
l)elow. 

Figure  37  .shows  the  time-domain  response  of  the  dynamical  system  based  on  the  logistic  mai), 
tlownsampled  by  a  factor  of  4.  Tonal  inputs  are  shown  along  with  the  system  respon.se.  \\  hen 
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Figure  37:  The  logistic  discrete^time  dynamical  system  is  downsampled  by  a  factor  of  4-  the 
top  figure,  a  tone,  shown  in  red,  drives  the  system,  which  exhibits  the  response  shown  in  green. 
Note  that  the  response  appears  random.  The  second  figure  shows  the  system  response  when  an 
ambiguous  tone  (one  who,se  downsampled  frequency  response  is  identical  to  the  first)  diives  the 
same  system.  The  system's  response  is  again  somewhat  random  and,  moi'e  significantly,  quite 
different  from  that  shown  in  the  top  figure. 


tonal  inputs  are  ambiguous  given  the  downsampling  factor,  the  sampled  systems  remain  (juite 
distinguishable.  Figure  38  shows  the  innovations  of  the  discrete-time  logistic  map  both  as  modeh'd 
and  under  the  linearized  model  required  for  compressed  sensing.  There  is  good  agreement  between 
the  two  models.  Figure  39  shows  the  successful  identification  of  superimposed  tones  for  the  logistic 
map  dynamical  system. 


4.2  RELATIONSHIP  OF  DYNAMICAL  SYSTEMS  TO  COMPRESSED  SENSING: 
CONTINUOUS-TIME  SYSTEMS 

Ordinary  Differential  Equations  Continuous-time  dynamical  systems,  as  studied  exten¬ 
sively  in  the  literature,  are  characterized  by  an  ordinary  differential  equation  of  the  form  x  =  f{x). 
In  analogy  to  the  discrete  case,  we  consider  a  driven  system  of  the  form 


x{t)  =  fixit))  +  y{t), 


(24) 
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Figure  H8:  The  logistic  discrete-time  dynamical  system  is  downsampled  by  a  factor  of  4-  Eight 
tones  randomly  located  act'oss  the  Imnd  dritre  the  system.  The  red  trace  shows  the  time  series  of  the 
driven  dynamical  system.  The  green  trace  shows  the  modeled  behavior  after  linearization.  Then' 
is  good  qualitative  agreement  between  the  true  and  linearized  models. 
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Figur'e  39:  The  logistic  diHcrete-time  dynamical  system  is  downsampled  by  a  factor  of  4^  Eight 
tones  randomly  located  across  the  band  are  successfully  identified  (green  tf'ace).  The  signals  occupy 
2  %  of  the  hand^  For  rompanson,  the  crosses  mark  the  locations  of  signals  identified  with  matching 
pursuits^  The  red  trace  shows  the  spectr'um  of  the  dowjisampled  data,  which  has  a  noise~like 
appemnnce.  The  black  t}ertical  lines  show  the  signal's  true  locations. 
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where  //(/)  represents  the  signal  input.  This  input  can  consist  of  known  s{t)  and  unknown  a{t) 
coinpoiK'iits  under  the  relationship  s{t)  =  s(f)  -h  rr(f).  To  simplify  the  discussion,  we  assume  that 
the  known  component  vanishes. 

Given  sampled  observations  of  the  state  vector  we  wish  to  determine  the  input  vector 
y{t).  This  can  be  accomplished  if  the  input  is  suitably  constrained.  For  example,  the  input  may 
be  l)andlimited,  as  assumed  below.  The  sample  rate  required  to  estimate  the  input  depends  on  the 
input  signal’s  bandwidth.  This  sample  rate  can  be  reduced  if  the  input  is  sparse  in  some  sense. 


We  sketch  one  path  toward  reconstruction  of  the  input  signal.  Consider  the  ancillary  au¬ 
tonomous  system  for  state  vector  x  over  the  inter\^l  [??T,  (r?  -f- 1)7")  given  the  initial 

condition  x{nT)  =  x(nT).  The  variable  T  indicates  the  time  between  sampled  values  x(7iT)  of  the 
dynamical  system.  This  autonomous  system  is  a  predictor  of  x{t).  The  discrepancy  e(t)  j'(f)— 7(/) 
between  x(t)  and  x(f)  measures  the  effect  of  the  input  y(t)  on  the  dynamical  system  during  the 
interval  [/jT,  (n  -f  1)T).  Let 

/;(nr)  =  liiii  j-(0  -  J-(f)  (25) 

so  that  /^(nT)  ran  be  interpreted  as  a  discrete-time  innovations  process. 


To  first  order,  f(f)  satistfies  a  linear  ODE  for  the  form 


e(/)  =  D/(x(f)Mf)  +  i/(0  (26) 

in  the  interval  [7fT,(n+  1)T)  with  the  initial  condition  e(nT)  =  0.  The  expression  Df(x{t)) 
represents  the  Jacobian  matrix  of  the  mapping  f{x)  evaluated  along  the  trajectory  x  of  the  predictor. 
The  values  of  x{t)  are  determined  by  the  initial  condition  x{nT)  =  x{nT),  which  is  the  last  sampled 
observation.  The  linearity  of  the  ODE  for  e{t)  allows  us  to  express  the  innovations  as  a  solution  to 
the  linear  algebra  system 

/  :  \ 


PiT)  \ 


f)(nT)  J 


=  A 


(Iti 


(27) 


■  / 


where 


^  -  m6)dT, 

JnT 

given  the  cardinal  series  representation  of  the  bandlimited  signal  s(t) 
•‘>■(0  =  51  ^’■’nP(t  -  mS) 


(28) 


(29) 


for  some  <5.  The  entries  A„,n  of  the  coefficient  matrix  A  are  shown  in  tlie  form  of  tfie  general  solution 
to  a  system  of  linear  ODE’s  given  inputs  of  the  form  p{t  —  niS).  The  notation  “T.O.”  indicates  a 
time  ordering  of  the  matrix  products  (integrating  factor),  which  is  required  when  the  ODE  system 
is  not  a  single  scalar  equation  (see  |18|  for  an  interesting  discussion  of  these  solutions). 

4.3  APPLICABLE  COMPRESSED  SENSING  ALGORITHMS 

In  evaluating  the  performance  of  compressed  sensing  for  dynamical  systems,  W'e  use  the  mixed 
L^^Li  norm  as  an  objective  function.  Specifically,  let  represent  a  known  m  x  n  real  matrix,  g  a 
length  m  observed  vector,  and  /  a  length  n  unobserved  vector.  The  objective  function 

IlfZ-slli  +  f-ll/lll  (30) 

is  minimized  over  /  to  obtain  a  sparse  solution  to  an  underconstrained  least-squares  fit  given  m  <  n. 
The  procedure  we  follow  is  based  on  [19]  and  |3).  Taking  the  gradient  of  the  objective  function  yields 

V||$/-ff|li  =  2^'^($/-s).  (31) 

If  the  shrinkage  function  5^  is  defined  on  n- vectors  as 

X  —  fi/2  X  >  /r/2 

X  +  p./2  X  <  —p/2  ,  (32) 

0  otherwise 

then  tlie  update  step  of  an  iterative  minimizer  for  equation  (30)  is  given  by 

+  (33) 

In  use,  the  iterative  step  is  initialized  with  /  =  0  and  iterated  a  fixed  number  of  times.  If  the 
update  f  is  still  zero  (as  is  often  the  case  when  p  is  large),  the  value  of  p  is  reduced  and  the  algorithm 
is  repeated  until  a  nonzero  /  is  reached.  To  ease  implementation,  it  is  useful  to  run  the  iterations 
on  a  low  dimensional  subspace  which  is  spanned  by  a  smaller  number  n'  <  n  of  components  of  /. 
These  components  can  he  selected  using  basis  pursuit  |20|. 

Another  type  of  approach  [21 1  involves  the  use  of  a  sequence  of  surrogate  functions  that  bound 
the  objective  function  and  are  easier  to  miiiiinize.  For  example,  the  L\  norm  has  quadratic  upper 
bounds  expressed  by  the  one-dimensional  inequality  (see  Figure  40) 

,34) 

.Added  to  the  quadratic  least-squares  term,  the  resulting  upper  bound  is  quadratic  and  can  be 
minimized  exjjlicitly.  Then  new  upper  bounds  can  be  determined,  tt  cetera.  The  resulting  iterations 
are  expressed  by 

/  -  +  /idiag(/)-’ )-'^^y.  (35) 
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Figure  4^:  The.  L\  nomi  in  one  dimension  c.a7i  be  upper- bounded  by  guadratics  that  are  tangent 
to  L  [  contours  at  the  coordinates  of  points  from  successive  stages  in  the  iteration.  Added  to  the 
quadratic  least-squares  iet'nu  the  resulting  upper  bound  on  the  objective  functioji  is  quadratic  and 
easily  minimized,  ptvviding  the  next  point  in  the  iteration  and  the  next  set  of  bounding  quadrntics. 
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in  the  loop. 

It’s  worth  noting  that  the  iterative  approach  in  equation  (33)  is  also  based  on  a  surrogate 
function.  In  this  case  the  surrogate  does  not  have  the  quadratic  term  contributed  by  the  least- 
squares  portion  of  the  objective  functioin  The  resulting  surrogate  can  be  optimized  component  by 
component  |19]. 


4A  EXAMPLES  OF  A  BROADBAND  IMPLEMENTATION 

4-4-1  Broadband  Circuit  and  Model 

A  broadbaml  microwave  dynamical  system  based  on  a  simple  nonlinearity  has  been  studied 
in  [22|.  This  <lynainicaJ  system  is  described  by  the  ldac:k  diagram  of  Figure  41,  w4iich  shows  a,  tent- 
map  nonlinearity  within  a  loop  with  delayed  feedback.  The  nonlinearity  is  ini[)lemented  as  shown  in 
Figure  42.  The  implemented  system  exhibits  a  bandwidth  on  the  order  of  500  MHz  given  feedback 
delays  of  about  10  nsec.  This  low-poww,  wideband  implementation  is  extremely  simple.  It  appears 
that  operating  bandwidths  can  be  extended  to  multiple  GHz  without  difficulty  by  miniaturizing  the 
circuitry. 


4-4.2  Phase  Space  Dynamical  Model 

The  dynamical  system  is  modeled  in  [22]  by  an  integro-differential  equation  for  the  voltage 
This  equation  incorporates  a  model  for  the  bandpass  response  of  the  feedback  loop.  Here  a 
source  s{t)  is  added,  resulting  in 


u(t)  -h  av 


v{l)dl  +  s{t)  =  dF{{cu{t  - 


(36) 


where  F(i;)  is  the  tent- map  nonlinearity  given  by 


F{v)  wo  -  sj Ff{v}  + 


(37) 
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Figure  42:  Circuit  diagram  of  the  tent-map  nonlinearity. 


and 


F,(tO 


( 


Ai{v  —  i/) 
Ar{v  —  V*) 


V  <  V* 

V  > 


(38) 


For  the  example  treated  here,  the  parameters  in  the  integro-ditferential  equation  bec'ome  «  =  A 
u;q/A,  and  d^=  a  =  9,  which  in  turn  depend  on  the  parameters 


= 

.47 

= 

-.(>2 

«>• 

= 

.05 

n* 

= 

.12 

= 

0 

7 

= 

3.7 

oJl) 

= 

1.510" 

A 

= 

2.5  10" 

(39) 


A  dc'tailed  physical  decription  of  the  circuit  model  and  interpretation  of  the  parameters  are  presented 
in  |22|.  It  suffices  to  note  here  that  the  loop  delay  is  expressed  by  S, 

There  are  several  ways  to  convert  the  integro-differential  equation  (36)  into  a  delay-differential 
ecjuation  (DDE).  One  is  pursued  here.  Defining  the  state  variables 


dof 

def 


T(f) 

y{t) 


v{t) 

v{t) 
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and  difFcrentiating,  we  have  the  DDE 

KO 

—y{t)la  —  bx{f)/a  —  s{t)Ju  +  dc.F{c.x{t  —  —  6)/a 

4.4.3  ODE  Approximation 
By  defining  auxilliary  variables,  a  DDE  can  be  a  approximated  by  an  ODE.  We  will  illustrate 

def 

the  modeling  process  for  the  DDE  of  equation  (40).  Let  Xk{f)  =  x{t  +  kSg)  and  similarly  for  yk{t)- 
.Assume  that  the  loop  delay  is  a  multiple  of  the  sample  interval  d,.  In  other  words,  assume  mSg  =  S. 
Then 


This  system  of  equations  in  the  augmented  state  variables  {(0:4.,  yn)}  is  certainly  not  finite.  The 
loop  delay  creates  an  unbounded  memory  in  the  DDE  system  that  is  not  modeled  exactly  with  a 
finite  system  of  ODEs.  Note  that  the  additional  state  variables  ^  system  with  a 

banded  coefficient  miitrix  that  contains  only  two  bands:  the  diagonal  and  a  single  off-diagonal  band 
reflecting  the  delay  coupling. 

Since  the  loop  gain  'y  limits  the  coupling  expressed  by  the  off-diagonal  terms,  it  is  plausible 
to  exi>ect  that  a  truncated  version  of  the  ODE  system  equation  (41)  can  approximate  a  solution  of 
the  DDE  equation  (40)  sufficiently  far  from  the  truncation  boundary. 

As  an  example  of  an  approximating  ODE  model,  truncate  equation  (41)  so  that  the  ODE 
has  2  X  100  state  variables  corresponding  to  100  sample  times  (indexed  by  k  above).  .Assign  the 
value  zero  to  coefficients  in  the  ODE  system  involving  vciriables  that  are  out  of  liounds.  Values  at 
the  sample  times  provide  initial  conditions  for  the  system  of  ODE’s,  which  are  solved  to  determine 
tlie  between-sainple  values  that  are  required  for  compressed  sensing.  The  level  of  effort  in  the 
computation  is  limited  by  the  very  sparse  nature  (2  bands)  of  the  system  matrix.  In  principle, 
solving  the  ODE  system  over  one  sample  interval  results  in  a  time  series  valid  over  100  intervals;  in 
practice,  the  ODE  only  approximates  the  solution  to  the  DDE.  .More  specifically,  the  initial  sample 
intervals,  which  are  near  the  truncation  boundary,  do  not  accurately  rnotlel  the  DDE  solution. 

The  difference  between  the  DDE  and  ODE  solutions  is  shown,  for  example,  in  Figure  43. 
Time,  spanning  100  saitiple  intervals,  is  labeled  on  the  x-cixis.  Relative  error  is  shown  on  the  y-axis. 
Note  that  the  ODE-modeling  error  is  very  large  for  initial  samples,  but  settles  rapidly  to  between 
-40  and  -50  dB.  The  latter  residtial  is  determined  by  the  accuracy  of  tlie  DDE  ami  ODE  integrators, 
which  can  be  tuned  to  some  extent. 
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Figure  The  DDE  system  can  be  approximated  by  an  ODE  system.  The  approximation  .suffers 
errors  due  to  tr'uncation  of  the  inherently  mfinite  dimensional  ODE  model  required  by  the  infi¬ 
nite  memoi'y  of  the  DDE  system.  Loss  due  to  truncation  is  shown.  Large  losses  occur  near  the 
tiuncation  boundai'y.  These  lo.s.ses  decrease  mpidly  as  time  progresses. 
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4.4.4  Linearization  of  ODE  Model 


Coefficient  matrices  relating  basis  expansions  of  signals  (e.g.,  cartlinal  series)  and  observations 
(e.g.,  innovations)  can  be  derived  for  ODE  models  of  dynamical  systems  using  the  linearization 
methodology  of  Section  4.2.  examine  linearization  for  the  ODE  model  of  the  DDE  system  of 
Section  4.4.2. 

Introduce  the  notation  xji.(#)  ‘=  Xjt(t)  +  and  yk{f)  =  yk{t)  +  <f>k(f)  where  (xA.(t),yA-(*)) 

satisfies  equation  (41)  with  s{t)  =  s{t).  Then  equation  (41)  can  be  linearized  about  the  solution  with 
input  s(t).  The  resulting  linear  ODE  with  input  .s(t)  =  s(f)  +o'(f)  and  state  variable  (xa-(0?<^a  (0) 
becomes 

( \  =  (  ^  f  ^ 

V  *•(<) )  V  -‘*’0  y  V  / 

(  0  0  \  ) 

—  +  kSt)  I  ?  j  ■  (■*2) 


The  coefficient  matrix  is  a  function  of  the  solution  {xk{t),yit{t)),  resulting  in  a  linear  ODE  with 
nonconstant  coefficients.  .-Igain,  the  linear  system  is  truncated  upon  application  to  the  DDE  system 
of  Section  4.4.2. 

4.4.5  Performance 

Some  brief  examples  of  performance  are  presented  for  the  DDE  system.  The  intent  is  to  illus¬ 
trate  qualitatively  the  performance  of  the  dynamical  system.  System  i)arameters  are  not  optintized 
in  any  significant  sense.  In  this  stage  the  results  are  primarily  anecdotal,  but  the  system  itself 
is  practical,  wideband,  low  power  and  capable  of  extension  to  larger  band  widths  on  the  order  of 
multiple  GHz. 

.\lthough  the  results  are  mainly  qvjalitative,  a  simple  quantitative  analysis  based  on  a  linearized 
system  model  is  presented  for  the  case  of  impulsive  inputs. 

The  input  to  the  dynamical  sy.stem  consists  of  the  combination  of  a  known  signal  (a  tone  in 
the  examples)  and  an  unknown  superposition  of  wideband  pulses.  The  pulse  bandwiths  are  1  GHz. 
The  tone  is  a  large  signal  that  is  used  to  ensure  sufficient  signal-dependent  randomization  even  if 
the  input  signals  are  weak.  The  system  is  downsampled  by  a  factor  of  3  to  5.  The  3  dB  bandwidtli 
of  the  dynamical  system  suggests  the  factor  of  three  downsampling,  but  the  slow  spectral  roll  off 
shown  in  Figure  45  makes  moot  the  choice  of  a  particular  downsampling  factor. 
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Figun^  44-  The  re.spome.  of  the  dgnamical  system  chosen  as  an  example  is  shown.  The  input  to  the 
dyjumiical  system  consists  of  three  imdebund  pulses,  none  of  which  are  apparent  in  the  system  's 
j^esponse. 


Figure  44  shows  tlie  timo-doiiiain  response  of  the  dynamical  system.  The  interval  shown 
in  the  figure  includes  all  three  wideband  iniimlses,  but  none  are  evident  in  the  plot.  Figure  45 
shows  the  spectrum  of  the  smiie  dynamical  system.  Note  the  prominent  discretes  in  the  wideband 
spectrum,  Tlie  c4iaracter  of  the  spectrum  is  highly  dependent  on  the  values  of  circuit  parameters 
and  in  particular  on  the  aniDimt  of  feedback. 

The  coefficient  matrix  describing  the  linearized  ODE  approximation  to  the  DDE  system  is 
based  on  equation  (42).  Graphicallyj  the  linearized  system  results  in  a  coefficient  matrix  with  the 
support  shown  in  Figure  46.  The  matrix  entries  are  based  on  a  cardinal  series  model  {train  of  fixed 
[julses  with  unknown  coefficients)  of  the  signals.  Note  that  the  coefficient  support  is  sparse  Eind 
banded,  reflecting  the  delay  structure  in  the  DDE. 

When  3  pulses  of  an  impulsive  waveform  are  used  to  drive  the  DDE,  compressed  sensing  a]>plied 
to  the  linear  ODE  model  can  be  used  to  recover  the  unknown  input  armed  only  with  knowledge  of 
the  signal  basis  (impulses,  in  this  example),  as  shown  in  Figure  47.  The  true  pulse  locations  are 
shown  with  vertical  lines  and  the  estimated  powers  are  shown  by  the  green  curves.  The  compressed 
sensing  techniciucs  used  for  recovery  are  described  briefly  in  the  beginning  of  Section  4.3. 

The  coefficient  matrices  used  for  signal  recovery  are  based  on  the  linearized  ODE  model  of 
the  DDE  system.  This  model  provides  accurate  predictions  of  system  response  that  can  be  used  to 
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Figure  ^5:  The  dynmniml  system  chosen  as  an  example  exhibits  the  spectrum  shown  here.  Note 
the  notch  at  low  frequencies  due  to  the  bandpass  character  of  the  circuit.  The  spectrum  is  wideband 
with  significant  discretes. 
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Figure  The  coefficient  matrix  used  to  determine  the  input  signal  is  shown  as  an  image,  re¬ 
vealing  the  typical  banded  structure  that  reflects  the  bulk  delay  in  the  DDE  (gaps  between  bands) 
as  well  as  fading  memory  refated  to  the  size  of  the  feedback. 

form  statistics  of  performance  under  various  densities  of  input  signals.  In  Figure  48,  the  cunuilative 
disti-ibution  of  the  loss  of  SNR  is  shown  under  various  system  loadings.  For  example,  with  1%  of  the 
time  line  ociuipied  by  wideband  (1  GHz)  pulses,  each  additional  wideband  pulse  suffers  very  little  loss 
associated  with  interference  from  other  pulses.  As  the  time  line  fills,  the  loss  becomes  significantly 
larger.  These  losses  do  not  include  the  typical  folding  loss  associated  with  downsampling,  which 
is  about  the  same  as  the  dowuisampling  factor.  This  loss  is  suffered  even  in  the  absence  of  other 
interference,  as  long  as  the  RF  device  noise  dominates  the  LSB  of  the  quantizer. 
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Figure  4  7:  Multiple  pulses  are  detected  successfully  after  downsampling  by  a  factor  of  3  to  5. 
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Figure  4S:  For  various  pulse  densities,  the  loss  in  SNR  incurred  due  to  downsampling  by  a  factor 
of  3  to  5  is  shown.  Background  pulses,  which  are  normally  welUseparated  fi'om  each  other  in  time, 
create  self-interference  within  the  dynamical  system.  The  amount  of  interference  depends  on  the 
downsampling  factor  as  well  as  the  fraction  of  the  time  line  that  is  filled.  This  phenomenon  occurs 
with  any  signal  basis.  Not  included  in  the  loss  is  the  folding  loss  as.sociated  with  downsampling. 
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5.  PERFORMANCE  BOUNDS 


Ill  this  section,  we  discuss  performance  bounds  for  the  techniques  presented  in  Sections  2  aiui 
3. 


Ill  Sections  5. 1-5.3,  we  present  and  derive  A2I  reconstruction  performance  bounds  for  BP  and 
\’PU,  respectively.  The  analog  receiver  configuration  for  the  scenarios  presented  are  illustratecl  in 
Figure  49  and  the  details  of  the  hardware  implementation  are  presented  in  |12|,  [23|  mid  Section 
5.1.  In  all  cases  and  with  no  loss  in  generality,  the  signals  are  subject  to  random  linear  projections 
by  circuitry  in  the  analog  RF  front-end  after  which  the  signals  are  digitized  using  an  undersampliiig 
analog- to-digital  converter  (ADC).  First,  the  rationale  and  requirements  for  random  linear  projec¬ 
tion  are  presented,  after  which  performance  bounds  are  derived  for  undersanipled  signals  both  with 
and  without  noise  present. 

In  order  to  understand  the  penalty  in  performance  associated  with  compressed  sensing,  in 
Section  5.4  we  examine  accuracy  of  the  estimates  of  some  signal  parameters.  Specifically,  we  consiiler 
qua.si-stat ionary  signals  with  unknown  covariance.  The  accuracy  of  covariance  estimates  is  evaluated 
under  the  assumption  of  a  sparse  signal  environment.  Accuracy  is  characterized  as  a  function  of  the 
amount  of  undersampling. 

Finally,  with  Section  5.6  we  tlescribe  the  performance  bounds  for  NoLAfF  hypothesis  testing. 


5.1  RANDOMIZATION  REQUIREMENTS 


Downsamjiling  induces  correlation  between  the  columns  of  the  basis  matrix  of  the  received 
signal.  This  can  be  expressed  as  D  =  »  where  the  downsample  matrix 


Hos 


1  0  •••  0  •••  0 

0  •••  1  0  •••  0 

•  :  10 


Rds  €  RLrf  'I' €  (43) 


selects  every  dth  element  from  each  column  of  the  basis  matrix  "F,  where  d  is  the  downsample  factor 
ami  the  symbol  [.rj  repre.sents  the  greatest  integer  that  is  less  than  or  equal  to  x.  .4s  an  example, 
consider  a  signal  that  is  sparse  in  an  orthonorinal  fretiuency  basis.  Each  of  the  columns  in  the  l)a.sis 
matrix  (i.e.,  IDFT  matrix)  after  downsampling  is  identical  to  d  other  columns  corresponding  to 
the  effects  of  aliasing.  .4n  ideal  anti-alia.sing  filter  prior  to  tlownsainpling  woukl  place  zeros  in  the 
alia.sod  signal  vector  locations.  That  is,  for  the  received  signal  vector 

y  =  H[}s'^iDF7\f'^  =  D/of"i'X  where  F  =  diag{  Is  Os  Is  ),  (44) 

Didft  ^ 
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Figure  49:  Block  diagram  of  the  analog  front-end  of  a  A2l  receiver.  The  bandwidth  of  the  received 
spectrum  is  undersampled  by  a  factor  of  d. 

columns  [^J  to  N  —  [^J  of  the  downsampled  IDFT  matrix  Dippr  do  not  contribute  to  the 
received  signal  vector  y  because  the  data  x  have  been  brick-wall  filteretl  by  the  diagonal  matrix 
F.  This  is  the  traditional  view  of  alias-free  downsampling,  where  the  brick-wall  filtered  baseband 
signal  is  ideally  confined  to  only  have  nonzero  support  that  spans  the  first  and  last  columns 
of  the  IDFT  matrix  idft-  Ifi  however,  the  signal  locations  are  unknown  at  the  receiver  and  not 
c-onfined  to  baseband,  brick-wall  filtering  will  not  facilitate  the  undistorted  recovery  of  the  signal  x 
from  the  downsampled  signal  y. 

A21  mitigates  the  effects  of  aliasing  with  random  linear  projections  prior  to  downsampling  |12|. 
The  key  idea  is  to  replace  the  downsample  matrix  Hps  with  a  randomizing  downsample  matrix  0 
given  by 

<*'1.2  •••  ‘*'i.rf+i  •••  0 

•••  <*’2,rf+i  ‘*'2.d+2  ••• 

(45) 

•  d  d  :  u;3.2d+l  a’3.2d-l-2 

0  0  0  0  0  0  0 

where  each  of  the  uJij  entries  in  the  matrix  above  is  an  i.i.d.  random  variable  drawn  from  a  Gaussian 
or  ±  Bernoulli  distribution  |24l;  e.g.,  see  Figure  49.  The  key  idea  is  that  if  the  dot  product  between 
the  columns  of  the  matrix  D  =  is  very  low,  then  in  principle  it  should  be  possible  to  recovi'r 
the  signal  x  in  the  basis  in  which  it  is  sparse  because  it  is  unlikely  that  an  element  from  x  will  be 
mistakenly  identified  with  more  than  one  column  of  D.  In  general,  the  number  of  random  nonzero 
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entries  in  each  row  of  0  needs  to  he  greater  than  or  equal  to  the  downsample  factor  d  to  ensure  that 
all  of  the  fnll-rate  sampling  phases  of  the  signal  are  represented  in  the  measurement  matrix  0\  that 
is. 


N 

d 


(46) 


where  the  notation  [X]i  represents  the  ith  column  of  matrix  X,  the  superscript  ^  is  the  matrix 
transpose  operator  and  ||x||()  is  the  so-called  ^o-norm  of  the  vector  x. 


5.2  A2I  VIA  BASIS  PURSUIT 

Constrained  /’i -programming  techniques  (basis  pursuit)  employed  in  compressed  sensing  are 
remarkably  efficient  in  identifying  and  reconstructing  signals  that  occupy  sparse  and  random  loca¬ 
tions  in  th('  orthogonal  basis  in  which  the  signals  are  compressible.  However,  for  signals  that  occupy 
any  more  than  a  very  small  fraction  of  contiguous  basis  positions,  the  basis  pursuit  performance 
bound  offers  no  guarantee  on  identification  and  rec’onstruction  performance.  In  the  following  para¬ 
graphs  we  will  outline  the  basis  pursuit  performance  bound  as  first  presented  in  |24|  and  |25|,  an 
then  offer  an  interpretation  of  the  bound  in  the  context  of  identifying  not-so-sparse  signals  at  the 
outj)ut  of  a  downsampling  A2I  receiver. 

Consider  a  sparse  signal  occupying  unknown  frequency  locations  in  some  bandwidth  D  that 
is  digitized  at  a  rate  2B  d.  The  digitized  signed  can  then  be  described  by 

y  =  Dx  +  V  (47) 

where  v  €  Rl-'rf  is  the  noise  with  power  inequality  E{v^ v}  <  r/  ,  and  E{-}  is  the  expectation 
o})erator.  In  |1|  it  was  demonstrated  that  it  is  possible  to  identify  and  reconstruct  the  sparse  signal 
X  by  solving  the  basis  pursuit  (BP)  problem  formulated  as 

min  ||j-||]  s.t.  ||y  -  Dx\\l  <  J].  (48) 

The  successful  recovery  of  j-  by  BP  is  guaranteed  |25]  if  the  following  conditions  of  the  matrix  D 
obeys  the  uniform  uncertainty  principle.  Defining  A  C  {1,  •  •  •  A^}  and  to  consist  of  the  columns 
of  D  inch'xed  by  A,  then  the  local  isometry  constant  S\{D)  is  the  smallest  number  satisfying 

il-S,(D))\\x.^g<\\[DUx^\\l<  (l+d-A(D))||xA||i,  (49) 

whore  x,\  €  and  the  glol)al  restricted  isometry  constant  is  defined  as 

rf,(D)4  sup  <)'a(£>)  (50) 

|A|=<7 
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with  |A|  denoting  the  cardinality  of  the  set  A  and  q  denoting  the  number  of  nonzero  terms  in  the 
vector  x.  One  can  show  that  it  is  possible  to  recover  an  estimate  of  signal  x  to  w'ithin  a  Eticlidean 
distance  no  greater  than  Crj  |24|  where  C  is  a  constant  if 

d3,(Z?)  +  ;«4,(D)  <2.  (51) 


An  interpretation  of  the  equations  (49)  through  (51)  above  is  that  if  the  downsampling  factor  d 
is  increased  multiplicatively  by  some  factor  q,  tlien  the  recovery  of  the  signal  x  to  within  some 
Euclidean  distance  Cij  is  now  only  guaranteed  if  x  spans  4a  fewer  positions  in  the  basis  .  As  an 
example,  consider  the  toy  problem  of  a  signal  received  without  noise,  i.e.,  ti  =  0,  where  it  should 
theoretically  be  possible  to  perfectly  recover  the  signal  x  if  the  conditions  as  expressed  in  (51)  are 
met.  With  A  denoting  an  eigenvalue  of  the  matrix  ,  then  from  (49)  it  is  easy  to  see 

that 


■  IkAll? 


<  A 


^max 


(52) 


SO  that  54g(Z>)  is  bounded  from  below  by  the  value  I  when  —  0  because  4^  >  [^J  which 

violates  the  inequaiity  of  equation  (51).  In  other  words,  even  with  no  noise  present  there  is  no 
guarantee  that  BP  can  identify  and  reconstruct  the  signal  x  if  the  number  of  nonzero  values  in  the 
basis  ^  is  greater  than  |  under  the  most  ideal  of  conditions.  As  we  will  shortly  show,  this 
will  be  a  limiting  factor  in  BP  identifying  and  reconstructing  signals  that  occupy  only  a  fraction  of 
the  locations  in  the  downsampled  Nycjuist  band. 


5.3  A2I  VIA  MAXIMUM  LIKELIHOOD  TECHNIQUES 

In  this  section  we  will  derive  a  nutxinium  likelihood  bound  for  identifying  and  reconstructing 
signals  that  occupy  a  much  larger  number  of  contiguous  basis  locations  than  BP  has  a  guaranteed 
bound  for.  In  doing  so,  we  will  set  the  stage  for  the  derivation  and  application  of  Variable  Projection 
and  Unfolding  presented  in  Section  3.1  to  not-so-sparse  signals  at  the  output  of  a  sub-Nyquist 
sampling  A2I  receiver. 

Let  s  =  where  Sj  e  {1/2,***  Si  ^  Sj,  and 

D  =[[i?]J  [/?y  where  [£)],  e  is  the  matrix  whose  columns  are  spanneti  by  the  signal (s) 

in  the  basis  passband^  with  >  g.  Then  the  received  signal  vector  can  be  expressed  as 

ij  =  [D]^x^  +  +  Dik  (53) 

where  g  ^  is  the  signal  energy  in  the  passband,  av  ^  is  the  signal  energy  in  the  stop 

band,  and  v  G  is  the  receiver  noise.  It  is  easy  to  see  from  equation  (53)  that  if  the  matrix  D  is 

'  Wc*  are  liberally  using  the  term  ^passband’  to  denote  the  locations  in  the  ba.sis  where  signal  energy  is  principally 
confined. 
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orthonornial,  i.e.,  perfect  mutual  incoherence  holds,  then  the  signal  Xa  niay  be  recovered  from  1/  in 
the  presence  of  noise  by  matched  filtering,  i.e.,  when  j/  =  +  .  However,  downsampling 

will  generate  a  matrix  D  with  more  columns  than  rows;  therefore,  all  of  the  columns  will  never  be 
perfectly  orthogonal.  In  |26|  and  |27|,  reconstruction  performance  is  directly  related  to  the  degree 
of  colierence  between  the  columns  in  D.  We  will  take  a  statistical  signal  processing  viewpoint  of 
reconstruction  performance  under  the  following  conditions:  the  noise  is  independent  and  normally 
distributed  as  v  ~  j\/'(0,  and  the  stopband  signal  energy  is  negligible,  e.g.,  »  0  .  Then  the 

maximum  likelihood  estimator  (MLE)  is  given  by 

= 

x,  +  v,+  ([D\«Cz'[D\,)  '\D\»CZ'\D\^V^,  (M) 

Foldiyig  Lobs 

where  the  covariance  Ci,  =  D{<tII)D^ .  Equation  (54)  consists  of  three  terms;  the  desired  signal, 
the  noise  at  the  locations  in  the  basis  that  the  signal  occupies,  and  a  folding  loss  whicJi  corresponds 
to  noise  outside  these  locations  folding  back  in.  The  folding  loss  is  the  penalty  that  is  paid  in 
A2I  for  using  reduced  rate  sampling  and  ML  reconstruction.  Notice  that  the  degree  of  the  folding 
loss  is  directly  related  to  the  coherence  between  the  columns  in  the  matrix  D-,  as  the  cosine  of  the 
angle  between  the  columns  of  [I>],  and  approaches  zero,  so  does  the  folding  loss.  The  MLE 
formulation  in  eciuation  (54)  may  significantly  ^unplify  some  of  the  noise  components,  .‘\nother 
cost  function,  the  minimum  mean  square  error  MMSE  |28|,  is  often  used  to  balance  the  quality  of 
the  estimate  against  the  potential  for  noise  amplification.  Using  equation  (53)  with  rj  0  mid 
Xa  ~  A/"{0,  Ca:,)  ,  e.g.,  the  signal  is  subject  to  frequency  selective  Rayleigh  fading,  then  the  M.MSE 
estimator  is  given  by 

=  (IClf  C-'  IBl,  +  C-')->  \D]«  c;'y.  (55) 

Using  the  matrix  inversion  lemma,  it  is  easy  to  show  that  (55)  is  equivalent  to 

Folding  Loss 

-  D-\b-^  +  Ca,X'D-^[DyjC-\\D]aXa  +  Dv),  (56) 

MM$E  Folding  Bias 


where  D  =  [£>jf  ’  [D]^  . 

The  folding  loss  associated  wdth  ML/MMSE  identification  and  reconstruction  is  illustratetl  in 
Figure  50. 

The  scenario  illustrated  in  Figure  50  employed  a  measurement  matrix  9  wdth  entries  randomly 
chosen  from  a  ±  Bernoulli  distribution  with  signals  whose  basis  coefficients  covered  random  but 
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ML/MMSE  Performance  Bound 


Figure  50:  Folding  loss  associated  with  an  ML/MMSE  receiver.  Note  that  noise  folding  loss 
corresponds  to  an  overall  loss  in  SNR  as  a  function  of  the  signal's  baridwidth  (sparse  in  frequency) 
and  the  downsampling  factor. 

contiguous  positions  in  frequency.  The  folding  loss  in  ML/MMSE  identification  and  reconstruction 
corresponds  to  the  loss  in  SNR  due  to  random  linear  projections  followed  by  an  undorsanipling  {sub- 
Nyquist)  ADC.  The  folding  loss  may  be  explained  in  two  ways:  first,  as  explained  previously,  there  is 
a  loss  due  to  noise  from  ambiguous  basis  locations  folding  back  onto  the  basis  locations  that  tlie  signal 
occupies.  This  is  reflected  by  the  fact  that  folding  loss  is  greater  when  the  downsampling  is  more 
aggressive,  e.g.,  from  a  factor  of  2  to  a  factor  of  8.  Second,  as  the  bandwidth  of  the  signal  increases, 
the  variability  in  the  frequency  response  of  the  random  linear  projections  makes  it  more  likely  that 
there  will  be  noise  amplification  in  the  identification  and  reconstruction  process.  This  is  illustrated 
in  Figure  50,  which  shows  that  as  the  bandwidth  increases,  noise  amplification,  as  reflected  by  the 
change  in  folding  loss,  also  increases.  The  choice  of  an  ML  or  MMSE  estimator  is  dependent  on 
the  signal  bandwidth;  at  one  extreme,  narrowdmnd  spikes  (tones)  are  best  reconstructed  using  an 
ML  estimator,  while  a  wider  band  signal  should  be  identified  and  reconstructed  using  an  MMSE 
estimator  to  obtain  the  best  trade  between  noise  amplification  and  reconstruction  performance. 


5,4  COVARIANCE  ESTIMATION 

The  ability  to  estimate  signal  parameters  is  often  well  characterized  by  Crarner-Rao  bounds. 
For  compressed  sensing  applications,  these  bounds  have  little  direct  relevance  since  estimation  occurs 
on  the  basis  of  a  single  vector  observation.  Mixed  signal  conqmnents  are  extracted  using  models 
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of  the  observations  tliat  rely  on  known  signal  bases.  When  more  data  are  available,  incoherent 
integration  of  the  additional  samples  can  support  better  estimation  of  signal  parameters  that  inchnle 
powers  an<l  even  signal  bases  themselves.  It  is  also  well  known  that  model-based  parameter  estimates 
are  sensitive  to  modeling  errors.  These  errors  can  have  less  deleterious  effects  on  signal  statistics 
such  as  covariances.  Covariances  support  estimates  of  signal  powers  and  signal  bases  (more  precisely, 
signal  subspaces). 


Below,  some  initial  results  on  the  estimation  of  signal  covariances  are  presented.  The  quality 
of  parameter  estimates  is  characterized  by  Cramer-Rao  bounds  on  estimation  error.  The  effct:ts  of 
different  amounts  of  signal  knowledge  are  discussed  as  is  the  impact  of  undersampling  on  estimator 
]>(‘rformance. 


Let  us  assume  that  the  Cramer-Rao  bound  is  formulated  using  a  known  signal  basis  with 
unknown  signal  powers.  The  model  of  the  data  is  expressed  by  the  length  m  observed  vector 
yk  =  AyZf,,  given  tlie  known  in  x  n  matrix  Ak  and  the  unobserved  data  of  length  n  expressed  by  the 
vector  Zk-  Here,  k  is  the  sample  number.  Each  samjile  vector  Zk  is  assumed  to  have  the  covariance  R. 
The  matrices  /Ijt  are  assumed  to  vary  randomly  from  one  vector  sample  to  the  next.  We  assume  that 
the  n  X  n  covariance  R  is  diagonal.  For  the  moment,  assume  in  addition  that  the  s  m  diagonal 
entries  associatixi  with  signals  are  representetl  by  known  (e.g.,  the  first)  s  diagonal  components  of 
R.  Let  Cjt  denote  the  matrix  whose  sole  nonzero  entry  is  unity  in  the  (j,  A')*^  component.  Then  t  he 
approximate  Cramer-Rao  bound  matrix  for  the  coordinates  {ejj}  becomes,  in  the  limit  of  large  m 
and  n 


21 


(57) 


where  t  is  the  number  of  vector  samples.  When  the  location  of  the  signal  components  is  unknown, 
the  result  changes  only  slightly,  resulting  in  the  approximate  Cramer-Rao  bound  matrix 


2/ 


(58) 


The  terminology  ‘‘approximate”  reflects  the  sparseness  of  the  true  cov’ariance  (s  m)  and  the 
asymptotic  nature  of  the  expressions  (large  n  and  m). 


A  few'  comments  on  these  results  are  in  order.  First,  it  is  clear  that  the  effect  of  dowmsampling 
the  data  by  a  factor  of  n/m  results  in  an  increase  in  the  required  sample  support  by  a  factor  of 
{n/m)^  in  order  to  maintain  the  same  level  of  performance.  Second,  performance  is  independent 
of  the  signal  basis,  as  long  as  this  basis  is  known  to  the  receiver.  Third,  the  additional  know'ledge 
of  the  active  signal  components  does  not  improve  estimates  of  signal  powers.  It  should  be  recalled 
that  these  results  assume  that  the  signal  model  is  sparse  (s  <§:  m). 


Other  as|)ects  of  covariance  estimation  can  also  be  treated.  For  example,  signal  bases  {'an 
lie  estimated.  These  estimates  can  be  showm  to  decouple  from  those  of  the  signal  powers.  A  brief 
treatment  of  basis  estimation  will  be  presented  in  future  drafts. 
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It  is  interesting  to  examine  the  effect  of  a  priori  information  on  the  performance  of  covariance 
estimation.  The  random  parameter  Fisher  matrix  129|  can  be  used  to  assess  tlie  benefits  of  additional 
information.  In  particular,  this  approar:li  can  be  applied  to  the  exponential  priors  associated  with 
L\  penalty  functions  used  in  conjunction  with  maximum  likelihood  parameter  estimators.  Results 
indicate  some  benefits  from  prior  information  in  the  sense  that  the  required  sample  support  can  be 
reduced  somewhat.  These  results  will  also  be  reported  in  a  future  draft. 

Cramer-Rao  Bounds  When  the  environment  is  (luasistat ionary  it  is  possible  to  character¬ 
ize  the  performance  of  certain  estimators  in  terms  of  Cramer-Rao  lower  bounds  on  mean-squared 
errors  of  parameter  estimates.  Specifically,  conskler  vector-valued  Gaus.sian  data  with  probability 
density  function 

p(2lR)  =  7r-"iRi-*e-"^'*“‘".  (59) 

We  assume  that  the  data  obeys  a  complex,  circular  Gaussian  distribution,  which  is  discussed  in  many 
references  {see  [30]  for  a  self-contained  discussion  of  these  statistics  as  well  as  complex  versions  of 
some  standard  distributions  required  bekw).  When  more  than  a  single  observation  of  a  is  available, 
the  covariance  R  can  be  estimated  and  used  for  detection,  identification,  and  reconstruction.  T!ie 
benefits  of  stationarity  are  significant.  For  example,  the  model-based  techniciues  used  with  a  single 
observation  can  be  sensitive  to  modeling  errors  that  are  less  of  a  problem  with  covariance- based 
estimators.  Sparsity  still  plays  a  role  in  characterizing  performance  and  in  the  structure  of  the 
estimator,  but  the  theory  is  less  developed  in  this  case. 

The  performance  of  covariance-based  estimators  is  characterized  below  in  terms  of  Cramer- 
Rao  bounds.  Sparsity  is  assumed  in  order  to  arrive  at  a  simple  apjjroximation  for  the  bounds.  Of 
particular  interest  is  the  loss  in  performance  due  to  downsampling  the  observations.  Some  variants  of 
the  bounds  are  discussed  as  they  relate  to  different  signal  hypotheses.  Although  Cramer-Rao  lower 
bounds  are  only  achievable  in  an  asymptotic  sense,  they  are  typically  approached  with  small  sample 
support  (i,e.,  small  numbers  of  observations)  given  good  estimators,  as  long  as  the  appropriate 
SNR’s  are  above  certain  thresholds.  Determining  these  thresholds  is  a  much  more  difficult  problem, 
but  for  many  practical  examples,  the  thresholds  are  in  operationally  interesting  regions. 

For  our  applications,  the  data  are  observed  indirectly  through  an  m  x  n  matrix  A  in  tlie  sense 
that  only  y  =  Az  is  observed,  then  the  Fisher  matrix  for  covariance  parameters  is  given  by  (defining 

Ra  ARA^) 

F  =  2/E[t.r(R4'R^R-‘R.4R^‘s/j/^)].  (GO) 

The  variable  /  denotes  the  number  of  vector- valued  observations.  The  notation  Ra  is  short  for 
AXA^  with  X  a  tangent  vector  to  R  {conceptually,  X  =  /?).  Note  that  the  (‘ovariances  R  lie  in  the 
real  dimensional  vector  space  of  Hermit ian  matrices  so  that  all  tangent  vectors  can  be  viewed  as 
elements  of  a  real  vector  space.  Expectation  (denoted  E['])  is  taken  over  the  random  variates  A  and 
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y.  The  distribution  of  ;</,  conditioned  on  .4,  is  given  by  p((/|i?,4)  in  the  notation  of  equation  (59). 
DoKning  and  noting  that  the  conditionaJ  expectation  of  is  mean-zero  complex 

circular  Gaussian  with  covariance  we  can  reexpress  the  Fisher  as 

2/E[  \^]]  =  2lE[iT{R^'kAR:^^RA)]-  (61) 

The  measure  on  tlie  m.  x  n  matrix  A  is  assumed  to  be  invariant  under  unitaries  acting  on  the  left  or 
right  of  A.  For  example,  we  can  assume  that  all  entries  of  A  are  i.i.d.  complex  circular  Gaussians 
witfi  unity  complex  covariance.  Define  T  {AA^Y^f^A.  We  assume  m  <  n  so  that  the  rows  of 
T  form  an  orthonormal  basis  for  the  row  space  of  A.  In  particular,  TT^  =  We  can  rewTite  the 
Fisher  in  terms  of  the  induced  measure  p  on  T  as 

(A',  r)/?  =  F(A,  K)  =  2/  J  ti[{TRT^)-\TXT^){TRT^)-\TYT^)]  dfi{T).  (62) 

This  integral  emphasize.s  the  fact  that  only  the  statistics  of  the  row'  space  of  A  matter  in  the  Fisher. 
Note  that  the  invariance  properties  of  fi  imply 

{UXU\UYU^)uwi  =  {X,  Y)n.  (6.3) 


.At  this  stage  w'e  assume  that  the  true  covariance  matrix  R  is  sparse.  Specifically,  sparseness 
is  a.ssumed  to  be  equivalent  to  a  rejiresentation  of  R  as 


given  the  s  x  ,s  matrix  C,  with  s  in.  Here  t/  is  a  unitary  matrix.  If  w^e  write  TU  =  (Tj  Ta)  so 
that  TRT^  =  I„,  -  TiCrj,  then 

(r/?r+)-‘  =  -  r, {€-'  +  r/r, )~’t/  (65) 

and  hence 


/^>(r/?rt)-'  >/„, -Pr,,  (66) 

where  P■|■^  =  T[{T^T])~^T^  is  the  projector  onto  the  column  space  of  T),  provided  s  <  in.  This 

inequality  can  be  used  to  bound  the  integrand  of  equation  (62),  obtaining 


tr(rA'r^rAr^)  >  tr({Ti?rt)^'(TATt)(TPTt)-i(T.Vr^)) 
=  tr({TArt)=^)  -  2tr(PT,  (TXT^)^) 

+  tr(Pr,  (rAr^)Pr,  (TAT^)) 

>  triiTXT^f)  -  2tr(/V,  {TXT^f). 
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Next,  we  use  the  bound 


E|tr(7V.  (rxr>f)l  <  ElXl„(X)tT(Pr, )]  =  =  0(i)  (67) 

m  77?. 

to  conclude  that 

{xy)R  =  (x,y');„  +  o(^).  (68) 

A  sharper  result  can  be  conjectured  based  on  the  weak  dependence  of  Fr,  and  (rXT^),  namely 
{X,yh  =  (X,V),„+0(^).  (69) 

77/ 

A  proof  seems  rather  technical  and  the  sharper  result  does  not  help  us  here. 

The  inner  product  (X,y}/„  =  {-V,  K)  is  invariant  under  unitaries: 

f/y[/^)  =  (X,  y).  (70) 


All  real,  symmetric,  invariant  inner  products  on  Hermitian  matrices  have  the  form 


trX 

(X,y)  =  atr[(X-—/„)(y 


n 


try  ,  „  ,  trX  try 

- /„  - 

77  77.  n 


(71) 


for  iionnegative  coefficients  a,  6,  This  fact  is  well  known.  For  coinjdeteness,  we  present  a  selT 
contained  argument  that  can  be  skipped.  The  slick  argument  notes  that  the  Hermitian  matrices 
of  trace  zero  are  orthogonal  to  the  matrices  that  are  multiples  of  the  identity  matrix  and  both 
subspaces,  that  of  trace  zero  matrices  and  that  of  multiples  of  the  identity,  are  separately  invariant. 
The  subspace  of  trace  zero  Hermitian  matrices,  when  multiplied  by  the  square  root  of  —1,  can  be 
identified  with  the  skew-adjoint  matrices  that  form  the  Lie  algebra  of  the  Lie  group  SU (n)  of  unitary 
matrices  with  unity  determinant.  The  action  l7(iX)[/^  of  5[/(7?)  on  its  Lie  algebra  is  the  so-called 
adjoint  representation,  which  is  known  to  be  irreducible  (no  nontrivial  invariant  siibspac:es).  For 
convenience  and  more  generality,  let  the  notation  g  ■  X  =  gX  denote  the  action  of  a  group  element 
g  on  a  vector  space  element  X.  In  our  context,  if  g  represents  the  unitary  (7,  then  gX  corresponds 
to  [/A"t/L  The  relationship  between  invariant  inner  products  and  irreducible  representations  is 
cemented  by  Schur’s  lemma  as  follows.  Assume  we  are  given  two  invariant  inner  products  Oa-  for 
k  =  L2.  Also  assume  that  the  k  =  2  inner  product  is  nondegenerate.  Then  there  exists  a  linear 
operator  0  such  that  (X,  *)]  =  {0(A),  But  (gX,  ^)i  =  (^(gA),  ^)2  and 


(ffX,  •),  =  =  (cf>(X),ff-^-}2  -  (S0{X),  Os, 


(72) 


where  the  first  and  last  equalities  follow  from  invariance  of  the  inner  products.  Since  the  A*  =  2 
inner  product  is  nondegenerate,  we  have  (/^(gX)  =  30(A).  By  Schiir’s  lemma  and  the  fact  that  the 
action  by  g  is  irreducible,  we  have  that  0  =  c/d.  In  other  words,  0  is  a  multiple  of  the  identity. 
Thus  (^‘)i  =  c{‘,  02'  The  nondegeneracy  of  tr{Ay),  restricted  to  traceless  Hermitian  matrices  is 
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easily  sliowii  and  this  fact  croinpletes  the  proof  that  tr(.Vy)  is  a  unique  invariant  inner  product  on 
tliese  matrices,  up  to  a  scalar  factor. 

The  unknown  coefficients  «,  b  can  be  identified  by  plugging  in  specific  inatrit:es  into  the  inner 
product.  By  letting  A'  =  /„,  we  find  that  b  =  m.  The  remaining  coefficient  is  more  difficidt  to 
iwaluate.  To  proceed,  let 


A  = 


A,  n\ 

0  ())' 


(73) 


with  scalar  Aj.  In  addition,  let  E„,  =  (/tti  anti  express  T  =  EmU  for  some  unitary  U. 

Tlien 


TXT'TYT^  =  E„,UXU^ElEj„UYU^El.  (74) 

Let  U  =  {Ui  U2)  with  U\  representing  the  first  column  of  U,  so  that  UX  =  (t^i  Aj  0„,„_i).  Then 


U^ElE„,U  = 


irlEl,E„,u^  uIeIem  \ 
u}^ElE,„Ui  uIeLeM  ) 


Putting  the  notation  together,  equation  (74)  can  be  written 
E„,Ui  XiUl  ElE,„  UiXiU^El 

with  trace  ||£^mf7i||‘*Aj^.  Thus  the  integral  equation  (62)  becomes 

21  j ir\{E,nUX){U^ElE,„U){XU^El)\d^aU  =  'llXl  J  \\EiUi\\U^^iU. 


(75) 


(76) 


(77) 


The  measure  on  tlie  unitaries  is  normalized  Haar  measure.  Conceptually,  if  we  form  an  n  x  n  matrix 
tilled  with  i.i.d.,  mean  zero,  complex  circular  Gaussian  random  variates,  then  the  Q  factor  in  a  QR 
decomposition  will  have  the  homogeneous  density  of  the  normalized  Haar  measure.  In  particular, 
the  first  column  of  [/  has  a  distribution  modeled  by  that  of  a  complex,  circular  Gaussian  random 
vector  of  length  ri  with  i.i.d.  components,  normalized  by  the  L2  norm  of  the  same  vector.  To 
evaluate  the  integral,  w'e  need  to  identify  it  with  the  complex  version  of  a  canonical  random  variate. 
Let  the  random  variables  {w*  }  have  i.i.d.  complex  circular  Gaussian  distributions  with  zero  mean. 
Then  the  random  variable  associated  with 


ES  kkl" 

is  a  tmnplex  l>eta  random  variable  with  tlensity  |30| 


(78) 


(79) 


77 


Furthermore,  l|£i,{/i|p  has  the  same  distribution  as  this  beta  variate  with  parameters  fi  =  m, 
u  =  n  —  rn,  as  the  unitary  U  varies  uniformly  over  all  unitaries  (normalized  Haar  measure).  Thus 


p  E[||£;1„i7i  Ij*®]  =  f  in,  n  -  m) 

J  (« + 

Finally,  we  use  the  value  of  p  to  solve  for  a: 

{X,  X)  =  X\p  =  aX'lil  -  -]  + 

n 

so  that 

n  /  m  \  m  (777.7?  —  1) 

o.  =  - -  [p-  ^  \  =  — 

n  —  I  \  n/  /  H' 

Summarizingj 


+  1) 


<^,K) 
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=  J  trlTXT^  Ty'T^]dp{T) 


l)Tn 

l)n 


_  m(Tnn-l)*_  r/ V  trX  r  \  /v-  trV  r  \1  i  trX  trV 


(80) 


(81) 


(82) 


Let  us  assume  that  the  Fisher  matrix  is  based  on  a  known  signal  basis  with  unknown  signal 
powers.  Without  loss  of  generality  (due  to  the  invariance  of  the  inner  product),  we  can  assume 
that  the  covariance  R  is  diagonal.  For  the  moment,  assume  in  addition  that  the  s  diagonal  entries 
associated  mth  signals  are  represented  by  the  first  s  diagonal  components  of  R,  If  ej^  denotes 
the  matrix  whose  sole  nonzero  entry  is  the  then  the  approximate  Fisher  matrix  for  the 

coordinates  {cjj}  becomes 


{ {^jj  f  ^kk ) )  —  2/ 


77?(n7n  —  1) 
77.^  (77-  +1) 


77(n  -  777  +  1)  +  1  f' 
77.(77m  -  1)  "  "J  ^ 


(83) 


where  is  the  vector  of  all  ones.  Let  777  =  \p.  and  77  —  Ai>.  Then  the  inverse  of  this  approximate 
Fisher  matrix,  in  the  limit  of  large  A  becomes 


1 


(84) 


When  the  location  of  the  signal  components  is  unknown,  the  argument  changes  only  slightly,  re¬ 
sulting  in  the  inverse  of  the  approximate  Fisher  (for  large  A) 


1 

Jl 


(85) 
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5.5  A  GENERALIZATION  OF  BASIS  PURSUIT 


5.5.1  Summary  of  Results 

Basis  pursuit,  in  the  noiseless  ca.se,  is  formulated  as  the  L\  minimization  problem 

min  ll/ll  1,  (86) 

which  can  be  solved  by  linear  programming.  If  <I>/  =  g  has  a  sufficiently  sparse  solution,  then 
basis  pursuit  will  find  that  solution,  avoiding  what  would  normally  be  an  intractable  combinatorial 
search.  A  degree  of  sparseness  that  guarantees  success  for  basis  pursuit  has  been  cal(*ulated,  in  an 
asymptotic  limit,  in  |31|. 

The  asymptotic  upper  bound  on  sparseness  is  pessimistic,  given  the  simulated  performance 
of  related  algorithms  in  noise.  Furthermore,  the  limiting  regime  where  the  bound  applies  does 
not  always  accurately  model  important  applications.  For  example,  consider  a  fixed  receiver  Imnd 
sparsely  occui)ied  by  finite  bandwidth  signals.  As  the  sample  support  increases,  we  would  like  to  hold 
the  number  of  signals  fixed  as  the  number  of  samples  per  signal  increases  due  to  increased  frequency 
resolution.  Instead,  the  asymptotic  bound  would  increase  the  number  of  signals  (to  maintain  fixed 
spectral  occupancy)  while  decreasing  the  bandwidth  of  individual  signals. 

A  modified  version  of  basis  pursuit  can  handle  a  fixed  number  of  finite  bandwidth  signals  in 
the  limit  of  increasing  sample  support.  .Abstractly,  we  partition  the  columns  of  the  pX  x  rnX  mat  rix 
0  in  blocks  of  length  A.  The  blocks  represent  response  vectors  associated  with  the  same  signal. 
W’e  can,  as  is  convenient,  interpret  /  as  a  m  x  A  matrix  with  block  the  row  fj..  Then  the 
modified  form  of  basis  pursuit  becomes 

min  ^  ||/ii,..||2.  (87) 

In  words,  tho  snin  of  the  norms  of  the  blocks  of  /  is  minimized  under  the  given  equality  constraint. 
The  objective  hmction  is  L2  within  blocks  and  Lj  between  them.  The  solution  can  be  found  by 
solving  a  quadratic  programming  problem. 

.A  potential  advantage  of  the  quarlratic  programming  approach,  in  applicable  signal  niodels, 
is  a  much  larger  bound  on  the  fraction  of  the  band  that  can  be  handled.  Shown  in  Figure  51 
is  the  fraction  of  the  band  that  can  be  perfectly  n^constructed  from  noiseless  observations  under 
two  different  types  of  signal  models.  The  random  model  comes  from  (31|.  It  is  valid  in  the  limit 
of  large  receiver  bandwidth  or  in  the  limit  of  vanishing  signal  bandwidth,  given  fixed  receiver 
bandwidth.  In  effect,  each  signal  is  sampled  once,  corresponding  to  the  basis  vector  associated  with 
the  signal.  A  more  common  situation  occurs  when  the  receiver  and  signal  bandwidths  are  fixed 
while  the  observation  time  increases.  In  this  case,  the  number  of  samples  per  signal  increases  with 
the  observation  time.  .As  a  result,  performance  improves  substantially  as  indicated  in  the  figure. 
The  band  occupied  by  the  signals  is  unknown,  so  reconstruction  remains  a  potentially  combinatorial 
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Signal  Model 


Random 

Allocated 


Figure  51:  Two  types  of  asymptotic  bounds  on  the.  fractional  band  occupancy  that  permits  perfect 
reconstruction  are  shown.  The  random  bound  is  consistent  with  signal  models  that  have  fixed 
bandwidth  in  the  limit  of  increasing  receiver  bandwidth,  or  with  models  that  have  decreasing  signal 
bandwidth  in  the  limit  of  increasing  ohserimtion  time  with  fixed  receiver  bandwidth.  The  allocated 
bound  pertains  to  signals  with  fixed  bandwidth  in  the  limit  of  increasing  observation  time  and 
fixed  receiver  bandwidth.  These  signals  can  be  viewed  as  allocated  to  certain  subbands  that  are  not 
known  a  priori.  With  the  allocated  model,  as  observation  time  increases,  so  does  the  number  of 
samples  per  signal.  In  contrust,  the  random  model  maintains  a  .single  sample,  per  signal  in  any 
limit. 

problem;  but  the  quadratic  program  of  equation  (87)  can  be  used  instead  of  a  combinatorial  search 
to  achieve  exactly  the  same  result  in  the  noiseless  case. 

At  the  end  of  Section  5.5.2  some  comparisons,  with  and  without  noise,  are  made  b('tween 
different  versions  of  basis  pursuit. 

5.5.2  Gory  Details 

The  proof  that  equation  (87)  can  recover  a  sufficiently  sparse  signal  follows  in  detail  the 
argument  of  |31|.  The  ancillary  results  needed  are  stated  below;  their  proofs  are  essentially  the 
same  as  the  special  cases  stated  in  |31|  and  are  not  presented  here.  We  will  show  that  ecpiation  (87) 
follows  as  a  consequence.  Since  the  argument  closely  mirrors  that  in  |31|,  we  will  use,  for  the  most 
part,  the  notation  in  that  paper. 
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All  vectors  (soiiietimes  called  blocked  vectors)  c  are  assumed  have  lengtli  mX.  Their  compo¬ 
nents  are  groupetl  in  consecutive  blocks  of  length  A.  It  is  convenient  to  index  the  components  as 
r-jfc,  where  j  is  the  block  number  and  A-  the  component  in  the  block.  The  notation  Cj  .  refers  to  the 
vector  of  length  A  consisting  of  the  components  in  the  f  ^  block.  A  block  subset  T  of  the  indices 

{1,2 . in}  is  a  subset  of  all  block  indices.  The  length  of  T  is  denoted  \T\.  If  $  is  a  pX  x  mX 

matrix,  the  notation  denotes  the  matrix  of  size  pX  x  |rjA  consisting  of  the  |r|A  columns  with 
l)lock  indices  in  the  subset  T.  Similarly,  ct  denotes  the  vector  of  length  jTIA  supported  on  the 
blocks  indexed  by  T.  Denote  by  b{j)  =  {(j  —  1)A  -l-  1, . . . ,  jA}  the  set  of  indices  in  the  l)li)ck. 
Lot  consist  of  the  colninns  of  in  the  block. 

The  Euclidean  inner  product  is  denoted  {•,  •)  with  associated  Eudidean  norm  ||  •  ||.  For  con¬ 
venience,  we  let  (u),  denote  the  vector  of  length  A  whose  components  consist  of  tlie  inner 

product  between  w  and  {•olumns  of  A  vector  c  is  said  to  have  block  support  T  if  its  non-zero 

components  exactly  span  the  blocks  indexed  by  T. 

Some  of  the  definitions  of  |31|  are  slightly  altered  to  accommodate  the  block  structure  of  the 
signals. 


Definition  1  Define  6s  as  the  infimium  of  all  6  satisfying 

(1  -  '>')lkil  <  <  (1  +  ‘'')lkil  {88} 

where.  T  is  any  block  subset  of  length  at  most  S  unde  is  a  blocked  vector  of  the  appi'opnate  size. 
Only  rases  where  Ss  <  1  are  of  interest. 


Definition  2  Define  0s.S*  infimium  of  all  0  .satisfying 

max  {^TC.^Ttr^)  <  i9||rj|||r'||  (89) 

where  T  (T^ )  is  any  block  subset  of  length  at  most  S  (S^). 


W  e  briefly  state  some  of  the  leiiiinas  of  |3l]  in  the  slightly  more  general  form  needed  below. 
The  second  lemma  follows  from  the  first  by  an  essentially  equivalent  argument.  For  convenience, 
we  let  Os  = 

Lemma  1  that  Ss  <  1-  There  exists  a  vector  w  of  length  niX  and  block  length  A  such  that 

=  Cj.,  /  €  T,  where  T  is  a  block  subset  consisting  of  at  most  S  blocks  and  c  is  a  blocked 
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vector  supported  on  T .  If  S  +  S'  <  m,  there  exists  an  exceptional  block  set  E,  with  block  length 
|£|  <  S',  such  that 


)l/2 

< 

M  < 


(1-<55)V^ 


j^TuE 


1  — 


Ikll, 


jeE 


A'lkl!- 


The  following  is  the  result  of  whittling  away  at  the  exceptional  set  above. 


Lemma  2  Assume  that  6s  +  &s,2S  <  !•  With  c  as  above,  there  exists  a  w  satisfying  {w,  =  cj., 

j  €T,  and 


llK^6(j))ll  < 


% 

(1  —  6s  — 


Ikll, 


J^T. 


To  see  an  inimecliate  consequence  of  the  last  lemma,  let  c  be  supported  on  T  witli  cj.  a  unit 
vector  for  each  j  €  T.  Then 


_ _ 

I— 6s  —  lisas 


(90) 


when  j  ^T. 


Assume  that  ^5  +  ds,S  +  ^sas  <  !•  Let  /  be  a  blocked  vector  with  support  T,  |r|  <  5, 
that  solves  ^f  =  g.  Let  f  solve  equation  (87).  Define  the  blocked  vector  u  with  support  T  so  that 
Uj.  =  /j/||/j.l|.  We  can  find  a  weight  wsuch  that  =  Uj.,  j  e  T,  and  with  IK'u',  5*6{j))||  <  1 

for  j  ^  T.  Then 


E  ii/l-ii 


=  Eii/>+(/i-/i->ii+EiWii 

jer  jiT 

a  E  “i-  •  W-  +  f'i-  -  M  +  E(’"’  *«i))  ■ 

jer  j^T 

=  Y.  ii/j- II + ^Mi))  •  Uj .  -  /j )  +  ^60'))  •  fj- 

jeT  jeT  j^T 

=  Ell/j  ll  +  («^’  E  ^bU)fj.-^rf7’) 

j^T  l<i<m 

=  Eii/i  II + 

jeT 

=  Ell/ill- 

j 


(91) 
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Sinre  /'  minimizes  o<iuation  (87)  im<ler  the  linear  constraints,  it  follows  that  each  inequality  must 
he  an  equality  and  hence  /j .  =  1)  for  j  T.  Thus  ^r{fr  —  It)  =  ^“‘1  hence  /  =  /'  since  $/  has 

a  trivial  null  space. 

To  complete  the  argument,  first  note  that  OgyS'  ^  ^S+S'  is  established  in  j31|  and  the  same 
I>roof  can  be  applit'd  to  the  block  versions  of  these  variables.  Next,  note  that  (31|  informs  us  that, 
in  our  context, 


Prob 

where  A,„ax 
as  A  — *  oc. 


sup  A,„ax  >  { 1  +  +  VpX  +  f) 

T:\r\=S 


•)  G> 


-pAt^/2 


(92) 


is  the  principal  eigenvalue  of  Since  Amax  <  1  +  (Js,  we  have,  with  high  probability 


6s  <  -1  +  (l  +  \/Wpy  •  (93) 

Let  <l  tlcnote  the  downsampling  factor  7n/p  and  r  =  S/m  the  fraction  of  the  Itand  occu])ied  by 
signals.  Define 


2 

p(r.  (1)  '=  ^  —  1  +  ^y/kr  -  \/d  +  1^  >  +  <52.9  +  ^.is-  (9-1) 

A  =  l 

In  the  limit  of  large  A,  a  loww  bound  on  the  fraction  of  the  band  r  that  can  be  recovered  by 
equation  (87)  is  determined  by  p{r,d)  =  1. 

It’s  worth  noting  that  the  fixed  bandwidth  per  signal  could  have  been  variable,  but  known 
in  advance.  In  general  terms,  the  procedure  equation  (87)  is  used  to  exploit  a  priori  structural 
knowledge  of  signals. 


5,5.3  The  Noisy  Case 

.Although  no  results  are  itresented  here,  a  potential  practical  algorithm  for  solving  equation  (87) 
can  be  leased  on  (xiuation  (33).  Define 


,  .  dff  /  -r 

-.(X)  =  I  „ 


2M 


||.r||  <  ///2 

for  each  block  r-omponent  x  of  a  blocked  vector.  Exteiul  this  definition  to  all  blocks  using 


def 


. r,„.)  =  (fl'ji(;ri.) . rr^(3rm)). 


(95) 


(96) 


Then 


/-6V(/  + $'^(5 -$/)). 


(97) 
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provides  a  modified  gradient  algorithm  for  the  mixed  norm  problem 

+  /<EllA-  II-  (98) 

k 

That  this  is  an  appropriate  modification  of  equation  (33)  for  the  new  mixed  norm  problem  (98)  can 
be  established  using  surrogate  functions  in  much  the  same  way  that  equation  (33)  is  derived  in  [19|. 
We  make  no  claims  about  convergence,  but  it  seems  reasonable  to  believe  this  is  not  a  problem. 

As  an  example  of  the  performance  of  equation  (98),  consider  a  comi)arison  with  the  usual 
form  of  basis  pursuit,  as  shown  in  Figure  52.  The  example  uses  100  samples  for  standard  basis 
pursuit,  which  utilizes  equation  (98)  for  blocks  of  size  one,  and  500  samples  for  generalized  basis 
pursuit  utilizing  blocks  of  size  5.  No  noise  is  present  in  this  example.  The  figure  demonstrates  that 
signals  represented  by  multiple  basis  vectors  with  a  known  pattern  can  potentially  occupy  larger 
fractions  of  the  band  than  randomly  located  signals,  each  represented  by  a  single  basis  vector.  The 
probability  of  identification,  in  both  cases,  refers  to  the  number  of  signals  correctly  detected  and 
located.  If  both  curves  are  based  on  500  samples,  the  standard  basis  pursuit  degrades  slightly 
while  the  generalized  version  is,  of  course,  unchanged.  The  latter  comparison  corresponds  to  the 
asymptotic  bounds  treated  above. 

When  noise  is  present,  the  general  conclusions  drawn  from  Figure  52  still  hold,  as  Figure  53 
illustrates. 


5.6  NOLAFF  HYPOTHESIS  TESTING  PERFORMANCE  BOUNDS 

NoLAff  hypothesis  testing  undersampling  performance  bounds  are  directly  related  to  the  per¬ 
formance  of  the  two  main  algorithmic  components  of  the  technique;  namely,  Bayesian  decision  which 
underlies  the  hypothesis  testing  to  resolve  ambiguity  and  iterative  nonlinear  equalization  (NLEQ) 
which  undoes  the  nonlinear  distortion  of  the  NoLAff  encoder.  We  note  that  while  performance 
bounds  for  nonlinear  processing  are  notoriously  hard  to  come  by,  we  can  make  a  few  simplifying 
assumptions  which  allow  us  to  ignore  this  factor  for  most  practical  implementations. 

In  the  low  SNR  input  signal  case,  the  nonlinear  artifacts  after  equalization  will  be  below  the 
noise  and  hence  ignorable.  At  the  other  extreme  of  v^ery  high  SNR,  we  can  safely  assume  that 
the  equalized  nonlinear  artifacts  are  significantly  lower  than  the  signal  of  interest,  albeit  possibly 
stronger  than  the  noise,  and  hence  not  a  limitation  to  the  ambiguity  resolution.  Assuming  j)erfect 
ambiguity  resolution,  residual  NLEQ  distortion  levels  are  typically  well  below  the  noise  level  of  the 
ADC  itself  (See  e.g.,  |32,33l). 

We  note  as  an  aside  that  only  the  nonlinear  artifacts  that  fall  onto  the  support  of  the  input 
signal  itself  concern  us  here  in  terms  of  linearization,  since  the  other  artifacts  (prior  to  linearization) 
are  in  fact  used  for  ambiguity  resolution.  Hence,  the  stronger  (but  still  linearizable)  the  nonlinear 
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Figuir  52:  The  standard  form  of  basis  pursuit  associates  one  basis  vector  to  each  signal.  When  the 
signals  air.  more  naturally  associated  with  multiple  basis  vectors  in  a  known  patteni,  moir  general 
versions  of  basis  pursuit  can  offer  better  performance  given  the  same  measure  of  sparsity.  Shown 
above  is  a  companson  of  two  cases:  one  with  a  single  basis  vector  for  each  signal  and  another 
with  5  for  each  signal.  By  u.sing  more  basis  vectors,  sparsity  can  be  reduced  by  a  factor  of  3  to  4 ^ 
The  example  has  no  noise. 
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Figiive  53:  The  i^iandard  fonri  of  basis  pursuit  associates  one  basis  vector  to  each  sigiiaL  When  the 
sig7ials  are  rnoT'e  naturally  associated  with  multiple  basis  vectors  in  a  known  pattern,  moir.  general 
versions  of  basis  pursuit  can  offer  better  perfortnance  given  the  same  measme  of  sparsity.  Shown 
above  is  a  comparison  of  two  cases:  one  with  a  single  basis  vector  for  each  signal  and  another 
tvith  5  for  each  signal.  By  using  more  basis  vectors,  sparsity  can  be  reduced  by  a  factor  of  S  to  4^ 
All  signals  have  an  SNR  of  30  dB. 
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distortions  are,  the  better  our  ambiguity  resolution  will  be.  In  practice  a  nonlinear  distortion  level 
of  10-15  dB  below  the  signal  is  invertible. 


Under  the  assumption  that  the  signal  can  be  recovered  up  to  its  undersampling  ambiguity 
with  SNR  that  is  at  least  as  good  as  that  imposed  by  the  .ADC  used  to  sample  the  NoLAfF-encoded 
rt'ceived  signal,  we  are  left  with  an  analysis  of  the  ambiguity  resolution  itself. 

Let  y  denote  the  measured  (undersampled)  nonlinear  distortions  of  the  signal  we  wish  to 
resolve.  That  is,  the  probe  and  its  nonlinear  artifacts  with  itself  and  the  received  signal  itself  are 
removed.  Let  z/’  denote  the  true  nonlinear  distortions  and  let  z^.i  denote  the  i’th  false  nonlinear 
distortions  signal,  that  is,  the  one  of  the  distortions  signals  due  to  an  aliased  copy  of  the  received 
signal.  A  bound  on  the  probability  of  error  is  found  from 


prob  (UAi )  <  ^  prob  (Aj) , 


(99) 


where  A,  is  the  event 


|y''zF..|  |y"zr| 
l|z/- .P  lIzrP  ■ 


(lOO) 
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6.  FUTURE  WORK 


The  topics  described  in  this  report  offer  new  and  innovative  approaches  to  problems  of  inin- 
iniizing  sample  rate  while  maximizing  the  information  output  of  receiver  systems.  In  this  section 
we  disc  uss  some  ('ontinuing  challengers  and  areas  for  future  research  and  development  leading  to 
realistic  implementations  of  these  techniques. 


6.1  FUTURE  WORK  FOR  NOLAFF 

6.1.1  Algorithmic  Development 

Although  NoL.Aff  hypothesis  testing  has  been  tested  in  various  simulations,  some  algorithmic 
c  hallenges  remain.  The  first  of  these  involves  the  overlapping  of  signal  information  onto  itself,  the 
probe,  or  other  signals  or  distortions  present  in  the  undersampled  spectrum.  Overlapping  degrades 
the  ability  of  NoL.\ff  to  create  and  disambiguate  hypotheses.  NoLAff  has  been  tested  as  a  means 
of  spreading  sparse  information  into  unused  basis  vectors.  While  some  overlap  can  be  c*urrently 
tolerated,  a  strategy  must  be  developed  for  all  signal  overlapping. 

Overlapping  of  information  may  arise  from  several  situations.  The  simplest  occurs  when  the 
linear  portions  of  the  signals  and  probe  are  nonoverlapping  in  the  undersampled  spectrum,  but  the 
nonlinearity  leads  to  cnerlapping  distortions  after  undersampling.  Figure  54  depicts  this  situation, 
with  yellow  blocks  indicating  frequency  locations  where  distortion  will  overlap.  More  difficult  sit¬ 
uations  can  occur  when  a  narrowband  signal  happens  to  cross  a  fold  location  in  the  full  frequency 
spectrum,  heading  a  portion  of  the  signal  to  fold  on  top  of  itself  in  the  undersampled  spec  trum. 
Similarly,  multiple  signals  with  clear  spectral  separation  in  the  full  spectrum  may  overlap  when 
undc^rsampled  because  of  aliasing.  Algorithmic  solutions  to  these  challenges  need  to  be  explored. 

Further  performanc*e  benefits  may  be  achieved  by  exploring  the  use  of  an  adaptive  probe. 
Figure  55  shows  an  adaptive  probe  could  be  used  to  avoid  a  problematic  scenario  of  overlapping 
distortions.  Aside  from  avoiding  overlap,  an  adaptive  probe  may  be  able  provide  other  performance 
improvements,  suc‘h  as  steering  distortions  into  more  advantageous  spectral  locations  to  provide  a 
c  learer  separation  between  hypotheses. 

.•Ml  NoL.-\ff  simulations  to  this  point  have  utilized  probe  signals  that  are  injectcxl  by  the  system 
designer  and  entirely  known  a  prioii.  However,  there  is  no  reason  that  the  probe  must  be  a  signal 
injected  by  the  system  designer;  a  very  strong  signal  within  the  detected  environment,  such  as  a 
television  broadcast  or  a  jammer,  may  also  serve  the  same  mathematical  role  as  an  injected  probe. 
Suc*c'essfully  implementing  Nc)L.\ff  with  an  environmental  probe  would  turn  traditionally  interfering 
signals  into  benefic*ial  sources  of  signal  diversity. 

.AnothcT  area  for  further  rc^search  lies  in  optimizing  the  nonlinearity  for  NoL.-\ff  encoding. 
More  modalities  that  are  created  by  the  nonlinearity  provide  more  diversity  in  the  spreading  of  the 
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Figure  54:  Oiwrlapping  distortioris  occur  with  NoLAff  information  spreading.  This  figure  graphi¬ 
cally  depicts  the  undersampling  matrix  the  DFT  dictionary  matrix  T,  the  nonlinear  spir^ading 
matrix  Nr,,  and  the  sparse  infoi'mation  vector  $.  The  spreading  matrix  is  shown  with  several 
signals  present  The  yeJlow  squares  represent  locMions  with  overlap.  As  more  distortions  are 
created^  filling  out  the  signal  spaces,  more  of  these  overlap  locations  begin  to  exist 

signal.  However,  as  more  spreading  occurs,  the  likelihood  of  overlapping  information  increases.  The 
encoding  filter  can  also  be  developed  to  emphasize  differences  between  hypothesis  signals  across  the 
full  signal  spectrunn 

6*1-2  NoLAfF  Mechanization 

This  report  demonstrated  the  feasibility  of  implementing  a  NoLAff  encoder  with  relatively 
simple  circuit  components.  Future  work  for  NoLAff  mechanization  would  involve  the  construction 
of  a  testbed  to  analyze  performance  from  actual  hai'dware  components.  Initial  encoding  tests  would 
verify  performance  with  smaller  bandwidths  using  the  circuits  described  in  this  report.  However,  it 
would  be  desirable  to  demonstrate  the  ability  of  sampling  a  sparse,  very  wide  band  environment, 
such  as  8  GHz  or  wider,  using  a  high-speed  COTS  analog-to-digital  converter.  In  order  to  perform 
such  tests,  the  analog  encoding  hardware  must  be  designed  to  handle  iiiudi  wider  bandwidths 
beyond  the  capabilities  of  tlie  circuits  presented  in  this  report. 

The  implementation  of  a  NoLAff  decoder  in  digital  hardware  should  also  be  explored.  The 
hardware  design  trade-offs  between  speed,  i>erformance,  bandwidth  and  hardware  capabilities  should 
be  analyzed. 

6-1,3  NoLAff  Applications 

NoLAff  iindersainpliiig  has  shown  very  good  performance  for  signals  sparse  in  frequency.  Ap¬ 
plying  NoLAff  techniques  to  signals  sparse  in  other  bases  would  be  an  interesting  and  logical  di¬ 
rection  for  further  research.  Developing  NoLAff  approaches  for  exploiting  sparsity  in  time  and 
.sparsity  in  time- frequency  are  two  such  possibilities.  NoLAff  demonstrated  promising  results  for 
large  dynamic  range  environments,  and  we  would  seek  out  applications  requiring  large  dynamic 
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Figure  55:  The  upper  diagram  represents  the  cfration  of  undersampled  encoded  sigjial  with  a 
single  probe.  The  probe  basis  vector  is  indicated  by  the  green  anow^  and  the  sparse  signal  vectors 
are  indicated  by  the  wd  arrows.  In  the  top  diagram^  the  first  signal  vector  contains  ox^erlapping 
distortions t  which  reduces  NoLAff  decoding  performance.  The  lower  diagram  shows  how  the  probe 
can  he  shifted  to  a  different  basis  vector^  resulting  in  the  overlapping  distortion  location  moving 
away  from  the  basis  xmitor  useA  by  the  sparse  signal. 
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BP  Queuing  VPU 


Start  with  basis  pursuit  as 
queue  to  VPU 

J3U 

isoiate  those  eandidate  basis 
iocatioos  for  VPU  search 

Run  VPU  to  identify  basis 
locations  and  reconstruct  signai 


Figure  56:  Basis  Pursuit  cuing  VPU  to  identify  and  reconsU'uci  a  signal  that  occupies  roughly  80 
percent  of  do^imsampled  frequency  band. 

range.  One  sueh  application  is  the  Aogle-of- Arrival  problem^  involving  space- time  sparseness  and 
requiring  good  performance  over  a  large  dynamic  range.  AoA  is  described  below,  and  it  is  noted 
that  CS  techniques  fail  at  certain  dynamic  range  levels.  NoLAff  may  provide  better  perh>rinance 
for  this  jirohlem. 


6-2  COMBINING  BP  AND  VPU 

Although  BP  is  not  well  suited  for  reconstructing  signals  that  occupy  any  more  than  15-21) 
percent  of  the  downsampled  basis,  it  can  be  used  as  cue  to  VPU  to  reconstruct  signals  that  can 
occupy  up  to  nearly  100  percent  of  the  downsampled  basis.  The  process  of  BP  cuing  VTU  is 
graphically  illustrated  in  Figure  56;  first  BP  roughly  identifies  the  basis  support  and  reconstitutes 
undesirable  spurious  signals.  When  VPU  is  cued  by  BP  as  illustrated  in  Figure  56,  VPU  not  only 
identifies  ainl  reconstructs  signals  that  are  present  with  very  high  fidelity  but  also  rejects  spurious 
signals.  Combining  the  computational  efficiency  of  BP  with  the  performance  of  VPU  yields  a 
powerful  algorithm  far  more  robust  than  either  VPU  or  BP  operating  independently. 

Both  BP  and  VPU  suffer  from  the  so-called  near-far  problem  in  whkii  signals  received  with 
high  SNR  mask  low  SNR  signals  from  being  detected  or  reconstructed  with  reasonable  fidelity.  For 
example,  in  angle-of-arrivaJ  applications,  ML  techniques  are  frequently  outperformotl  by  subspace 
techniques  such  as  root-music  which  are  much  less  sensitive  to  power  variation  between  two  received 
signals.  However,  it  may  be  possible  to  recast  both  VPU  and  BP  in  a  subspace  frame?work;  fi)r 
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example,  consider  the  following  fornmlation  Y  =  AX  +  V  where  1/(^2) -  — .v(^7  )}r 

that  the  BP  prohleiii  can  he  formulated  as 

arg  mill  ||  V'  -  AX  fj+X\\  xt^  || ,  ^  ^  ^  ^  ^ 

X(^  ^  {.ri .  .rj,  ...,XN  }an(i  Xf  =  [[xi ,  Xg, . . . ,  xjv  II2, 

where  Y  is  the  received  signal  matrix,  A  the  randomized  downsampled  sparsity  basis  and  X  is  the 
signal  matrix.  The  intuition  behind  the  formulation  above  is  that  if  one  can  formulate  the  ^2  problem 
in  matrix  fornu  the  prol>lem  can  be  solved  using  subspace  techniques.  For  example,  consider  tlie 
case  where  the  S\^D  of  T  is  given  by  UDV^^  and  letting  Ysv  =  y  ^id  =  A'  then 
tlie  BP  problem  can  be  formulated  as 

argiiiiii  liy^v  -  AXsv  11^  +  A  ||xf  , ,  (102) 

where  [V]^,  are  the  singular  vectors  principally  associated  with  the  signal  subspat:e.  This  approach 
reduces  the  variation  of  the  noise  eigenvalues  and  may  mitigate  the  near-far  problem  faced  b>-  both 
BP  and  VPU. 


6*3  ANGLE-OF-ARRIVAL  IN  ARRAY  PROCESSING 

It  is  possilde  to  architect  a  unique  i^patial  compressed  sensing  receiver  consisting  of  an  array 
of  anteima  elements  that  sample  the  signals  arriving  at  the  face  of  the  array  from  a  countably  finite 
number  of  direc^tions.  The  spatial  A2I  receiver  is  illustrated  in  Figure  57,  where  an  r28-element  array 
is  broken  up  into  32  subarrays  consisting  of  4  elements  per  subarray.  The  output  of  eacli  (dement  is 
subject  to  randoin  linear  projections  witli  coefficients  drawn  from  a  ±  Bernoulli  distributiom  This 
type  of  compressc^d  sensing  receiver  a]jplies  random  linear  projections  and  then  downsamples  in  the 
spatial  (as  opposed  to  temporal)  domain. 

We  tested  the  performance  of  BP  in  an  environment  with  two  wideband  CDMA  signals  present, 
r(’ceived  with  an  SNR  of  5  dB  per  channeh  In  all  cases  we  compared  BP  performance  to  root-music 
operating  with  an  analog  front-end  receiver  almost  identical  to  the  receiver  illustrated  in  Figure  57, 
with  the  one  exception  that  the  root- music  array  was  steer€^d,  i.e.,  the  random  linear  projections  in 
Figure  57  were  replaced  by  complex  exponentials.  BP  was  modified  for  super- resolution  capability; 
this  involvfxl  refining  the  columns  of  ^  to  super-resolve  the  source  after  an  initial  identification  of 
the  coarse  spatial  basis  support  .  The  perfonnaiice  of  super- resolution  BP  as  compared  to  root-music^ 
is  illustrated  in  Figure  58.  We  found  that  in  general  super-resolution  BP  was  extremely  competitive 
with  root- music,  and  in  many  cases,  BP  slightly  outperfornmd  root-music  in  determining  tlie  AoA 
(in  azimuth)  of  the  two  vsources  present. 

We  also  compared  the  perfonnaiiee  of  BP  and  root- music  when  the  two  signal  sources  had 
unequal  signal  power,  and  the  results  of  this  test  are  illustrated  in  Figure  59.  With  one  source  40 
dB  stronger  than  the  other=the  so-called  near-far  problem — BP  was  unable  to  uniquely  identify 
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Subarray^ 


Figure  57:  Subarray  A 21  receiver  for  angle-of-arrival  processing. 

the  AoA  of  the  second  weaker  source.  With  an  array  calibration  error  model,  we  are  currently 
investigating  the  possibility  of  employing  estimation  and  subtraction  to  remove  the  effect  of  tlie 
strong  signal,  as  well  as  linear  projections  that  place  the  strong  interferer  into  the  null  space  of  D. 


6.4  DYNAMICAL  SYSTEMS 

A  key  aspect  of  future  endeavors  is  the  continued  study  of  dynamical  systems  (DS)  for  A2T. 
We  have  established  the  utility  of  DS  for  compressed  sensing  at  GHz  bandwidths  with  practical, 
low-pow’er  circuits.  What  remains  is  establishing  a  robust  modeling  approach  for  such  circuits  and 
assessing  the  impact  of  modeling  errors  on  performance. 


6.5  DENSER  ENVIRONMENTS 

Sparse  environments  often  have  additional  structure  that  can  l)e  exploited  to  enhance  the  per¬ 
formance  of  A2I.  For  example,  signals  can  have  allocated  bandwidths,  a-s  considered  in  Section  5.5. 
It  is  also  possible  for  signals  to  be  differentially  sparse  in  the  sense  that  their  environment  is  partially 
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Figure  58:  Angle-of- Arrival  processing  from  a  128-ele.ment  array  broken  up  into  32-element  sub- 
airays.  A 21  AoA  supei'-resolution  via  modified  BP  performance  is  compared  to  root-music  wheir 
in  the  case  of  root-music  each  of  the  subarrays  is  steered  to  point  at  a  specific  angular  location. 
Two  sources  located  at  -0.3827  and  0.7071  are  present. 


SNRs  of  60dB  and  20dB;  rootmusic  power  ignored 


Figwe  59:  Near-far  problem  as.sociated  with  BP  processing.  With  one  source  having  a  four  orders 
of  magnitude  higher  power  with  lespect  to  the  .second,  the  second  sowee  becomes  undetectable. 
Note  that  root-music  is  insensitive  to  the  near-far  problem. 
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known.  There  are  obvious  extensions  of  compressed  sensing  to  this  case  as  well  as  other  approaches 
that  may  even  be  useful  in  dense  environments  if  detection  alone  is  satisfactory. 

6.6  IMPROVED  DYNAMIC  RANGE 

Finding  low-level  signals  in  the  presence  of  strong  interferers  is  one  of  the  weaknesses  of  the 
standard  random  linear  model  of  compressed  sensing.  Some  implementations,  such  as  NoLAff,  can 
avoid  this  problem.  It  may  also  be  possible  to  use  more  robust  estimation  techniques  to  mitigate 
the  dynamic  range  problems  associated  with  conventional  compressed  sensing  algorithms. 
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7.  SUMMARY 


III  this  report,  we  presented  significant  developments  on  three  Analog-to-Inforination  (A2I) 
research  activities  at  the  heart  of  the  collaborative  effort  of  the  MIT  Lincoln  Laboratory  and  GMR 
R(‘search  &  Te(*hnology  team:  Nonlinear  Affine  processing  (NoLAff),  Variable  Projection  and  Un* 
folding  (VPU),  and  dynamical  systems.  Substantial  effort  has  been  made  to  demonstrate  the  practi¬ 
cal  capabilities  of  these  new  techniques  in  realistic  signal  environments  and  to  provide  a  path  toward 
mechanization.  We  presented  results  indicating  strong  potential  for  NoLAff  hypothesis  testing,  a 
novel  technique  invented  by  the  team  which  exploits  nonlinear  distortions  as  a  new  form  of  signal 
diversity,  enabling  undersampling  and  digital  reconstruction  of  sparse  signals.  NoLAff  showed  very 
accurate  performance  on  both  sparse  and  non-sparse  signals,  even  detecting  and  reconstructing  sig¬ 
nals  with  bandwidth  approaching  the  entire  undersampled  spectrum  with  less  than  a  0.1%  error 
rate.  Environments  with  multiple  signals  requiring  a  large  dynamic  range  (>40  dB)  were  shown  to 
be  (effectively  undersampled  and  reconstructed  with  NoL.Aff  as  well.  .A  candidate  analog  circuit  for 
.NoL.Aff  encoding,  robust  to  implementation  variations,  was  presented  along  with  simulation  n'sults 
that  suggest  a  hardware  NoL.Aff  system  may  realistically  perform  very  well.  The  complexity  of  the 
NoL.Aff  hypotliesis  digital  decoder  was  also  investigated  and  shown  to  be  reasonably  low. 

A  new  approach  to  solving  the  .\2I  problem  was  presented  in  Variable  Projection  and  Unfolding 
(V'PU).  \TU  is  similar  to  Orthogonal  .Matched  Pursuit  (OMP),  where  both  capture  the  most 
important  information  from  a  sparse  signal  by  using  a  subspace  containing  a  small  number  of 
random  linear  projections  from  the  original  signal  spa('e.  \TU,  however,  is  not  restricted  to  rank- 
1  subspa(‘es  and  can  exploit  correlations  between  coefficients  in  the  signal  basis.  .A  dual-stage* 
search  proc(*ss  accurately  locates  the  mostly  likely  position  of  the  downsampled  sp(H‘trum  within 
the  full  spc'ctrum.  This  process  showed  excellent  performance  on  signals  considered  too  dense  for 
standard  CS  or  Basis  Pursuit  (BP)  reconstruction,  signals  with  bandwidths  at  nearly  100%  of 
the  undersampl(*d  spectrum.  \TU  also  showed  good  performance  in  reconstructing  signals  sparse* 
in  bas(*s  other  than  fn'quency.  .Aj)plied  to  a  chirp  signal  buried  in  interference  and  noise,  \TU 
consistc'iitly  identifi(*d  the  undersampled  signal  correctly. 

Our  work  on  dynamic'al  systems  offers  a  unique  mechanization  approac'h  for  comprc'ssc'd  sens¬ 
ing.  Dynamical  systems  inherently  provide  the  randomness  and  memory  needed  for  undersampling 
with  CS,  and  allow  reconstruction  of  undersampled  signals  by  applying  sampled  values  as  initial 
conditions  to  an  ordinary  differential  equation  characterizing  the  system.  We  demonstrate*!!  the 
capability  of  this  dynamical  system  approach  to  CS  in  recovering  an  undersampled  pulse  signal 
using  a  j)ractical,  low-power,  wideband  circuit.  Nonlinear  models  were  formu!at(*d  to  support  the 
application  of  dynamic'al  systems  to  CS. 

The  work  resulting  from  this  ('ollaborative  effort  has  advanced  the  understanding  of  new 
and  unique  technologies  for  the  .A2I  probhun.  We  presented  groundbreaking  research  into  NoL.Aff 
processing,  and  NoL.Aff  hypothesis  testing  in  particular,  and  this  program  has  led  to  a  bett('r  under- 
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standing  of  the  capabilities  of  this  technology.  Our  results  suggest  NoLAff  may  out])erfonn  other 
A2I  techniques,  particularly  in  sparseness  and  dynamic  range.  VPU  is  also  an  original  technology, 
developed  through  A2I  research,  with  promising  results  for  both  non-sparse  signals  and  signals  with 
non-frequency-based  sparseness.  This  report  provides  a  strong  indication  that  these  technologies 
may  hold  practit^al  benefit  for  the  future  of  ELINT  and  SIGINT  systems  and  ISR  sensor  networks, 
for  both  detection  and  reconstruction,  by  offering  dramatic  improvements  in  the  ability  to  process 
information  more  quickly  and  with  greater  accuracy. 
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